2024-04-16T15:30:11.414 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-16T15:30:11.414 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi --owner scheduled_leonidus@teuthology --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} --verbose --archive /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189 -- /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/orig.config.yaml 2024-04-16T15:30:11.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T15:30:11.547 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189 branch: wip-lusov-shared-log-segment description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} email: null first_in_suite: false job_id: '7658189' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-shared-log-segment ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds debug zombie log segments: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Replacing daemon mds - \(MDS_INSUFFICIENT_STANDBY\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(MDS_DAMAGE\) - \(FS_DEGRADED\) sha1: c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: disabled: true install: ceph: flavor: default sha1: c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-shared-log-segment sha1: c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f owner: scheduled_leonidus@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 2402 sha1: c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f sleep_before_teardown: 0 subset: 111/111111 suite: fs suite_branch: wip-lusov-shared-log-segment suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEaLmnEMWE3/rOPSlclcAOSvPglNANZHh+V2bulxFaEAE5+8jQnVt2k/Pp0+fCPEMyOyYW5z+k5SEF+qNhZsT3s= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATWxCMAJge6GKTvl/K5DiJMRc7inzSVqkifT11tLL0Jml0L4eXlHil1S18hIMOzySdalstAjYssYekXInVLTrU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph config set mgr mgr/crash/warn_recent_interval 0 - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_failover teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-16_12:36:42 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-16T15:30:11.547 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa; will attempt to use it 2024-04-16T15:30:11.548 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks 2024-04-16T15:30:11.548 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-16T15:30:11.550 INFO:teuthology.task.internal:Checking packages... 2024-04-16T15:30:11.573 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f' 2024-04-16T15:30:11.573 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-16T15:30:11.574 INFO:teuthology.packaging:ref: None 2024-04-16T15:30:11.574 INFO:teuthology.packaging:tag: None 2024-04-16T15:30:11.574 INFO:teuthology.packaging:branch: wip-lusov-shared-log-segment 2024-04-16T15:30:11.574 INFO:teuthology.packaging:sha1: c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f 2024-04-16T15:30:11.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-shared-log-segment 2024-04-16T15:30:11.829 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2868.gc3c87ed0 2024-04-16T15:30:11.832 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-16T15:30:11.838 INFO:teuthology.task.internal:no buildpackages task found 2024-04-16T15:30:11.838 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-16T15:30:11.867 INFO:teuthology.task.internal:Saving configuration 2024-04-16T15:30:11.885 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-16T15:30:11.891 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T15:30:11.917 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 15:25:05.484096', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEaLmnEMWE3/rOPSlclcAOSvPglNANZHh+V2bulxFaEAE5+8jQnVt2k/Pp0+fCPEMyOyYW5z+k5SEF+qNhZsT3s='} 2024-04-16T15:30:11.940 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 15:25:05.485855', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATWxCMAJge6GKTvl/K5DiJMRc7inzSVqkifT11tLL0Jml0L4eXlHil1S18hIMOzySdalstAjYssYekXInVLTrU='} 2024-04-16T15:30:11.940 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-16T15:30:11.947 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-16T15:30:11.947 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-04-16T15:30:11.947 INFO:teuthology.run_tasks:Running task console_log... 2024-04-16T15:30:12.044 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c14b0b160>, signals=[15]) 2024-04-16T15:30:12.044 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-16T15:30:12.050 INFO:teuthology.task.internal:Opening connections... 2024-04-16T15:30:12.050 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:30:12.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T15:30:12.143 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:30:12.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T15:30:12.234 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-16T15:30:12.241 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-04-16T15:30:12.259 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-04-16T15:30:12.259 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-04-16T15:30:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-04-16T15:30:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T15:30:12.317 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T15:30:12.317 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T15:30:12.317 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-04-16T15:30:12.336 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-04-16T15:30:12.353 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-04-16T15:30:12.354 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-04-16T15:30:12.414 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-04-16T15:30:12.414 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-04-16T15:30:12.414 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-04-16T15:30:12.414 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-04-16T15:30:12.414 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-04-16T15:30:12.414 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-04-16T15:30:12.415 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T15:30:12.415 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-04-16T15:30:12.415 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-04-16T15:30:12.415 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T15:30:12.415 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-04-16T15:30:12.415 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T15:30:12.415 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T15:30:12.415 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T15:30:12.415 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-04-16T15:30:12.452 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-16T15:30:12.463 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-16T15:30:12.485 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-16T15:30:12.485 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-04-16T15:30:12.488 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-04-16T15:30:12.504 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-16T15:30:12.512 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-16T15:30:12.512 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-04-16T15:30:12.544 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-04-16T15:30:12.574 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-16T15:30:12.631 INFO:teuthology.run_tasks:Running task kernel... 2024-04-16T15:30:12.645 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-16T15:30:12.645 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-04-16T15:30:12.645 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-16T15:30:12.645 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T15:30:12.646 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T15:30:12.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:30:12.662 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-04-16T15:30:12.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:30:12.664 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-04-16T15:30:12.722 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-361.el9.x86_64 2024-04-16T15:30:12.722 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.14.0-361.el9.x86_64 2024-04-16T15:30:12.722 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-04-16T15:30:12.723 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-361.el9.x86_64 2024-04-16T15:30:12.723 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.14.0-361.el9.x86_64 2024-04-16T15:30:12.723 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2024-04-16T15:30:14.763 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-16T15:30:14.977 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-16T15:30:16.154 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-04-16T15:30:16.910 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - BaseOS 2.5 MB/s | 2.0 MB 00:00 2024-04-16T15:30:17.095 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-16T15:30:18.938 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-16T15:30:19.485 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.0 MB 00:02 2024-04-16T15:30:21.378 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-16T15:30:23.030 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-16T15:30:24.431 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-04-16T15:30:25.551 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-16T15:30:26.528 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-04-16T15:30:28.154 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-16T15:30:30.665 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-16T15:30:32.274 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 21 MB 00:03 2024-04-16T15:30:33.030 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-16T15:30:33.030 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-16T15:30:33.118 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-16T15:30:33.125 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-16T15:30:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-16T15:30:33.125 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-16T15:30:33.125 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-16T15:30:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-16T15:30:33.125 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-16T15:30:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-16T15:30:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-16T15:30:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-16T15:30:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:30:33.126 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-16T15:30:33.126 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-16T15:30:33.126 INFO:teuthology.orchestra.run.smithi071.stdout:Install 4 Packages 2024-04-16T15:30:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:30:33.127 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 77 M 2024-04-16T15:30:33.127 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 123 M 2024-04-16T15:30:33.127 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-16T15:30:33.417 INFO:teuthology.orchestra.run.smithi071.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 956 kB/s | 277 kB 00:00 2024-04-16T15:30:34.892 INFO:teuthology.orchestra.run.smithi071.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 8.6 MB/s | 15 MB 00:01 2024-04-16T15:30:35.800 INFO:teuthology.orchestra.run.smithi071.stdout:(3/4): kernel-modules-core-5.14.0-438.el9.x86_6 12 MB/s | 28 MB 00:02 2024-04-16T15:30:36.125 INFO:teuthology.orchestra.run.smithi071.stdout:(4/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 11 MB/s | 34 MB 00:02 2024-04-16T15:30:36.128 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-16T15:30:36.129 INFO:teuthology.orchestra.run.smithi071.stdout:Total 26 MB/s | 77 MB 00:03 2024-04-16T15:30:36.129 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-16T15:30:36.510 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-16T15:30:36.510 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-16T15:30:37.314 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-16T15:30:37.350 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-16T15:30:37.351 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-16T15:30:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-16T15:30:39.704 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-16T15:30:39.704 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-16T15:30:39.796 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-04-16T15:30:39.802 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-04-16T15:30:39.802 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-04-16T15:30:39.802 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-04-16T15:30:39.802 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-04-16T15:30:39.802 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-16T15:30:39.802 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-04-16T15:30:39.802 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-16T15:30:39.803 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-16T15:30:39.803 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-16T15:30:39.803 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:30:39.803 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-04-16T15:30:39.803 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-04-16T15:30:39.803 INFO:teuthology.orchestra.run.smithi122.stdout:Install 4 Packages 2024-04-16T15:30:39.803 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:30:39.803 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 77 M 2024-04-16T15:30:39.804 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 123 M 2024-04-16T15:30:39.804 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-04-16T15:30:39.974 INFO:teuthology.orchestra.run.smithi122.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.6 MB/s | 277 kB 00:00 2024-04-16T15:30:40.260 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-16T15:30:40.352 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T15:30:40.457 INFO:teuthology.orchestra.run.smithi122.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-04-16T15:30:41.166 INFO:teuthology.orchestra.run.smithi122.stdout:(3/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 25 MB/s | 34 MB 00:01 2024-04-16T15:30:41.558 INFO:teuthology.orchestra.run.smithi122.stdout:(4/4): kernel-modules-core-5.14.0-438.el9.x86_6 18 MB/s | 28 MB 00:01 2024-04-16T15:30:41.561 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-04-16T15:30:41.561 INFO:teuthology.orchestra.run.smithi122.stdout:Total 44 MB/s | 77 MB 00:01 2024-04-16T15:30:41.561 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-04-16T15:30:41.969 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-04-16T15:30:41.969 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-04-16T15:30:42.850 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-04-16T15:30:42.851 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-04-16T15:30:43.131 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T15:30:43.274 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T15:30:45.180 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-04-16T15:30:45.793 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-16T15:30:45.877 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T15:30:48.014 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T15:30:48.208 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:30:48.620 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T15:30:48.767 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T15:30:52.945 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:30:53.464 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T15:30:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:30:58.295 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:22.348 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:22.349 INFO:teuthology.orchestra.run.smithi071.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-16T15:31:22.349 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:31:22.391 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:22.724 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:22.724 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-16T15:31:22.724 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T15:31:22.724 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T15:31:23.144 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:23.144 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:31:23.144 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-16T15:31:23.144 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-16T15:31:23.145 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-16T15:31:23.145 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-16T15:31:23.145 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-16T15:31:23.145 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:31:23.145 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-16T15:31:23.462 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T15:31:23.563 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T15:31:23.563 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T15:31:23.563 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-16T15:31:23.563 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T15:31:23.564 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T15:31:23.564 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-16T15:31:23.564 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T15:31:23.564 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T15:31:23.643 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T15:31:23.643 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T15:31:23.643 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-04-16T15:31:23.677 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-04-16T15:31:23.677 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-04-16T15:31:23.677 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-04-16T15:31:23.677 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 68.el9 2024-04-16T15:31:23.677 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-04-16T15:31:23.677 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-16T15:31:23.677 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8264254 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-16T15:31:23.678 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-04-16T15:31:23.679 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-04-16T15:31:23.679 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:31:23.679 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-16T15:31:23.679 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-16T15:31:23.679 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-16T15:31:23.679 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-04-16T15:31:23.679 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:31:23.679 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-04-16T15:31:23.680 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-16T15:31:23.680 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 5.14.0-438.el9.x86_64 2024-04-16T15:31:23.680 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-16T15:31:24.282 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-04-16T15:31:25.569 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-16T15:31:25.615 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-04-16T15:31:25.617 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-04-16T15:31:25.631 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.VxWHetUNG7 2024-04-16T15:31:25.632 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VxWHetUNG7 2024-04-16T15:31:25.700 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.VxWHetUNG7 2024-04-16T15:31:25.839 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.VxWHetUNG7 is 6KB 2024-04-16T15:31:25.891 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.VxWHetUNG7 2024-04-16T15:31:25.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-04-16T15:31:25.970 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-16T15:31:25.970 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-16T15:31:25.970 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-16T15:31:25.970 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-16T15:31:25.974 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-16T15:31:26.141 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-04-16T15:31:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:27.555 INFO:teuthology.orchestra.run.smithi122.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-16T15:31:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:31:27.599 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:28.006 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:28.007 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-16T15:31:28.007 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-16T15:31:28.007 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:31:28.426 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-04-16T15:31:28.750 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T15:31:28.851 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T15:31:28.852 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T15:31:28.852 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-16T15:31:28.852 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-16T15:31:28.852 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-16T15:31:28.852 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-04-16T15:31:28.852 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-04-16T15:31:28.853 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-04-16T15:31:28.934 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-16T15:31:28.934 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-16T15:31:28.934 DEBUG:teuthology.orchestra.run.smithi122:> sudo rpm -qi grub2-tools 2024-04-16T15:31:28.976 INFO:teuthology.orchestra.run.smithi122.stdout:Name : grub2-tools 2024-04-16T15:31:28.977 INFO:teuthology.orchestra.run.smithi122.stdout:Epoch : 1 2024-04-16T15:31:28.977 INFO:teuthology.orchestra.run.smithi122.stdout:Version : 2.06 2024-04-16T15:31:28.977 INFO:teuthology.orchestra.run.smithi122.stdout:Release : 68.el9 2024-04-16T15:31:28.977 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: x86_64 2024-04-16T15:31:28.977 INFO:teuthology.orchestra.run.smithi122.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-16T15:31:28.977 INFO:teuthology.orchestra.run.smithi122.stdout:Group : Unspecified 2024-04-16T15:31:28.977 INFO:teuthology.orchestra.run.smithi122.stdout:Size : 8264254 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:License : GPLv3+ 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Packager : builder@centos.org 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Vendor : CentOS 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Summary : Support tools for GRUB. 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Description : 2024-04-16T15:31:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:31:28.979 INFO:teuthology.orchestra.run.smithi122.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-16T15:31:28.979 INFO:teuthology.orchestra.run.smithi122.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-16T15:31:28.979 INFO:teuthology.orchestra.run.smithi122.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-16T15:31:28.979 INFO:teuthology.orchestra.run.smithi122.stdout:hardware devices. 2024-04-16T15:31:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:31:28.979 INFO:teuthology.orchestra.run.smithi122.stdout:This subpackage provides tools for support of all platforms. 2024-04-16T15:31:28.980 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-16T15:31:28.980 INFO:teuthology.task.kernel:Updating grub on smithi122 to boot 5.14.0-438.el9.x86_64 2024-04-16T15:31:28.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-16T15:31:29.520 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-04-16T15:31:30.896 INFO:teuthology.orchestra.run.smithi122.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-16T15:31:30.926 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-04-16T15:31:30.928 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-04-16T15:31:30.943 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.stDMowvICT 2024-04-16T15:31:30.943 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.stDMowvICT 2024-04-16T15:31:31.012 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.stDMowvICT 2024-04-16T15:31:31.157 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.stDMowvICT is 6KB 2024-04-16T15:31:31.206 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.stDMowvICT 2024-04-16T15:31:31.220 DEBUG:teuthology.orchestra.run.smithi122:> sudo /bin/ls /boot/loader/entries || true 2024-04-16T15:31:31.285 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-16T15:31:31.285 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-16T15:31:31.285 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-16T15:31:31.285 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-16T15:31:31.287 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-16T15:31:31.438 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-04-16T15:31:56.170 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-16T15:31:56.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T15:31:56.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T15:32:01.446 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-16T15:32:01.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T15:32:01.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T15:32:56.227 DEBUG:teuthology.orchestra.remote:timed out 2024-04-16T15:33:01.455 DEBUG:teuthology.orchestra.remote:timed out 2024-04-16T15:33:05.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T15:33:05.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T15:33:05.614 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-16T15:33:06.059 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T15:33:06.059 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-438.el9.x86_64"... 2024-04-16T15:33:06.059 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-04-16T15:33:06.074 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-438.el9.x86_64 2024-04-16T15:33:06.074 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-16T15:33:06.074 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T15:33:06.075 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-16T15:33:07.076 DEBUG:teuthology.parallel:result is None 2024-04-16T15:33:10.456 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T15:33:10.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T15:33:11.951 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-16T15:33:12.385 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T15:33:12.385 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-438.el9.x86_64"... 2024-04-16T15:33:12.386 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-04-16T15:33:12.400 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-438.el9.x86_64 2024-04-16T15:33:12.401 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-16T15:33:12.401 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T15:33:12.401 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-16T15:33:13.401 DEBUG:teuthology.parallel:result is None 2024-04-16T15:33:13.401 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-16T15:33:13.409 INFO:teuthology.task.internal:Creating test directory... 2024-04-16T15:33:13.409 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T15:33:13.412 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T15:33:13.428 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-16T15:33:13.453 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-16T15:33:13.504 INFO:teuthology.task.internal:Creating archive directory... 2024-04-16T15:33:13.504 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T15:33:13.507 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T15:33:13.541 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-16T15:33:13.553 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-16T15:33:13.554 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T15:33:13.582 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T15:33:13.662 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T15:33:13.695 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T15:33:13.702 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T15:33:13.760 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T15:33:13.761 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-16T15:33:13.769 INFO:teuthology.task.internal:Configuring sudo... 2024-04-16T15:33:13.769 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T15:33:13.771 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T15:33:13.802 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-16T15:33:13.811 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-16T15:33:13.812 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T15:33:13.844 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T15:33:13.859 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T15:33:13.947 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T15:33:14.000 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T15:33:14.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T15:33:14.133 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:33:14.133 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T15:33:14.205 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T15:33:14.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T15:33:14.302 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T15:33:14.379 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T15:33:14.427 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:33:14.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T15:33:14.497 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-04-16T15:33:14.499 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-04-16T15:33:14.550 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T15:33:14.588 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T15:33:15.050 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-16T15:33:15.057 INFO:teuthology.task.internal:Starting timer... 2024-04-16T15:33:15.058 INFO:teuthology.run_tasks:Running task pcp... 2024-04-16T15:33:15.107 INFO:teuthology.run_tasks:Running task selinux... 2024-04-16T15:33:15.153 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-16T15:33:15.153 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-04-16T15:33:15.218 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-04-16T15:33:15.220 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-04-16T15:33:15.267 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-04-16T15:33:15.269 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-16T15:33:15.269 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-04-16T15:33:15.295 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-04-16T15:33:15.295 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-04-16T15:33:15.310 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-04-16T15:33:15.310 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive'} 2024-04-16T15:33:15.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T15:33:15.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:33:15.367 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T15:33:15.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:33:15.396 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-16T15:33:15.396 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-04-16T15:33:15.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-04-16T15:33:15.481 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-16T15:33:15.493 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-16T15:33:15.499 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-16T15:33:15.629 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-16T15:33:15.645 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-16T15:33:15.646 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-16T15:37:30.686 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-04-16T15:37:30.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T15:37:30.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T15:37:30.782 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-16T15:37:30.857 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-16T15:37:30.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T15:37:30.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T15:37:30.947 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-16T15:37:31.021 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-16T15:37:31.021 INFO:teuthology.run_tasks:Running task clock... 2024-04-16T15:37:31.034 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-16T15:37:31.034 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T15:37:31.034 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T15:37:31.037 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T15:37:31.037 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T15:37:31.066 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T15:37:31.084 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T15:37:31.093 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T15:37:31.110 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T15:37:31.114 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-04-16T15:37:31.128 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-04-16T15:37:31.140 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-04-16T15:37:31.146 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T15:37:31.155 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-04-16T15:37:31.163 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T15:37:31.172 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T15:37:31.189 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T15:37:31.221 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-04-16T15:37:31.226 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T15:37:31.226 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-16T15:37:31.226 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:37:31.226 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:37:31.226 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:37:31.227 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:37:31.244 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-16T15:37:31.247 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T15:37:31.247 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-16T15:37:31.247 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:37:31.247 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:37:31.248 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:37:31.248 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:37:31.249 INFO:teuthology.run_tasks:Running task install... 2024-04-16T15:37:31.257 DEBUG:teuthology.task.install:project ceph 2024-04-16T15:37:31.257 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f'}} 2024-04-16T15:37:31.257 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f'} 2024-04-16T15:37:31.257 INFO:teuthology.task.install:Using flavor: default 2024-04-16T15:37:31.263 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-16T15:37:31.263 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-16T15:37:31.264 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f', 'tag': None, 'wait_for_package': False} 2024-04-16T15:37:31.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f 2024-04-16T15:37:31.266 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f', 'tag': None, 'wait_for_package': False} 2024-04-16T15:37:31.267 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f 2024-04-16T15:37:31.408 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-shared-log-segment/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/centos/9/flavors/default/ 2024-04-16T15:37:31.408 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2868.gc3c87ed0 2024-04-16T15:37:31.487 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-shared-log-segment/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/centos/9/flavors/default/ 2024-04-16T15:37:31.487 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2868.gc3c87ed0 2024-04-16T15:37:31.522 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-shared-log-segment/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-shared-log-segment/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-shared-log-segment/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T15:37:31.522 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:37:31.522 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T15:37:31.553 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T15:37:31.553 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T15:37:31.604 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-shared-log-segment/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-shared-log-segment/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-shared-log-segment/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T15:37:31.604 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:37:31.604 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T15:37:31.623 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T15:37:31.633 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T15:37:31.633 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T15:37:31.702 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T15:37:31.705 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T15:37:31.773 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-04-16T15:37:31.774 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-04-16T15:37:31.784 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T15:37:31.852 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-04-16T15:37:31.853 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-04-16T15:37:32.035 INFO:teuthology.orchestra.run.smithi122.stdout:74 files removed 2024-04-16T15:37:32.063 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T15:37:32.101 INFO:teuthology.orchestra.run.smithi071.stdout:74 files removed 2024-04-16T15:37:32.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T15:37:32.696 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 254 kB/s | 83 kB 00:00 2024-04-16T15:37:32.932 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 167 kB/s | 83 kB 00:00 2024-04-16T15:37:33.146 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-16T15:37:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-04-16T15:37:33.522 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-16T15:37:33.783 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-04-16T15:37:34.033 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-04-16T15:37:34.244 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-04-16T15:37:35.683 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-04-16T15:37:35.786 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-04-16T15:37:36.610 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-16T15:37:36.698 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-16T15:37:38.469 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-16T15:37:38.525 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-16T15:37:42.634 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-16T15:37:42.678 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.2 MB 00:00 2024-04-16T15:37:44.418 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-16T15:37:45.014 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-16T15:37:46.388 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-16T15:37:49.231 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 21 MB 00:04 2024-04-16T15:37:51.463 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-16T15:37:54.168 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T15:37:54.169 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T15:37:54.182 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T15:37:54.183 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T15:37:54.185 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T15:37:54.186 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T15:37:54.188 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T15:37:54.191 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T15:37:54.192 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T15:37:54.269 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-16T15:37:54.270 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 6.6 k 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 5.4 M 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 890 k 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 145 k 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 1.5 M 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 142 k 2024-04-16T15:37:54.292 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 3.5 M 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 7.4 M 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 49 k 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 10 M 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 49 M 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 264 k 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 762 k 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 225 k 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 27 k 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 18 k 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T15:37:54.293 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 32 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 746 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 127 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 161 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 320 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 298 k 2024-04-16T15:37:54.294 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 99 k 2024-04-16T15:37:54.295 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 85 k 2024-04-16T15:37:54.295 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 3.1 M 2024-04-16T15:37:54.295 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 170 k 2024-04-16T15:37:54.295 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T15:37:54.295 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-04-16T15:37:54.295 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T15:37:54.295 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T15:37:54.295 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 3.4 M 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 3.1 M 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T15:37:54.296 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 21 M 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 34 k 2024-04-16T15:37:54.297 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 2.4 M 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 249 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 4.7 M 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 17 M 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 16 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 25 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T15:37:54.298 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 163 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 500 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 5.2 M 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T15:37:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T15:37:54.300 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 45 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 135 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T15:37:54.301 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T15:37:54.302 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T15:37:54.303 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T15:37:54.304 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:37:54.305 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-16T15:37:54.306 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-04-16T15:37:54.306 INFO:teuthology.orchestra.run.smithi071.stdout:Install 150 Packages 2024-04-16T15:37:54.306 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 24 Packages 2024-04-16T15:37:54.306 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:37:54.306 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 228 M 2024-04-16T15:37:54.306 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-16T15:37:55.414 INFO:teuthology.orchestra.run.smithi071.stdout:(1/174): ceph-19.0.0-2868.gc3c87ed0.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-16T15:37:55.764 INFO:teuthology.orchestra.run.smithi071.stdout:(2/174): ceph-base-19.0.0-2868.gc3c87ed0.el9.x8 9.3 MB/s | 5.4 MB 00:00 2024-04-16T15:37:55.814 INFO:teuthology.orchestra.run.smithi071.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 2.8 MB/s | 145 kB 00:00 2024-04-16T15:37:55.948 INFO:teuthology.orchestra.run.smithi071.stdout:(4/174): ceph-fuse-19.0.0-2868.gc3c87ed0.el9.x8 1.6 MB/s | 890 kB 00:00 2024-04-16T15:37:55.998 INFO:teuthology.orchestra.run.smithi071.stdout:(5/174): ceph-mds-19.0.0-2868.gc3c87ed0.el9.x86 13 MB/s | 2.4 MB 00:00 2024-04-16T15:37:56.290 INFO:teuthology.orchestra.run.smithi071.stdout:(6/174): ceph-mon-19.0.0-2868.gc3c87ed0.el9.x86 16 MB/s | 4.7 MB 00:00 2024-04-16T15:37:56.657 INFO:teuthology.orchestra.run.smithi071.stdout:(7/174): ceph-mgr-19.0.0-2868.gc3c87ed0.el9.x86 2.1 MB/s | 1.5 MB 00:00 2024-04-16T15:37:57.008 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T15:37:57.009 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T15:37:57.022 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T15:37:57.024 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T15:37:57.025 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T15:37:57.026 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T15:37:57.028 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T15:37:57.030 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T15:37:57.031 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T15:37:57.110 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-04-16T15:37:57.135 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-04-16T15:37:57.135 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-04-16T15:37:57.135 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-04-16T15:37:57.135 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-04-16T15:37:57.135 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T15:37:57.135 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 6.6 k 2024-04-16T15:37:57.135 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 5.4 M 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 890 k 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 145 k 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 1.5 M 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 142 k 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 3.5 M 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 7.4 M 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 49 k 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 10 M 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 49 M 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 264 k 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 762 k 2024-04-16T15:37:57.136 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 225 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 27 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 18 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 32 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 746 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 127 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T15:37:57.137 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 161 k 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 320 k 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 298 k 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 99 k 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 85 k 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 3.1 M 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 170 k 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T15:37:57.138 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 3.4 M 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 3.1 M 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T15:37:57.139 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 21 M 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 34 k 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 2.4 M 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 249 k 2024-04-16T15:37:57.140 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 4.7 M 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 17 M 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2868.gc3c87ed0.el9 ceph-noarch 16 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 25 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 163 k 2024-04-16T15:37:57.141 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 500 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 5.2 M 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T15:37:57.142 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 45 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.0.0-2868.gc3c87ed0.el9 ceph 135 k 2024-04-16T15:37:57.143 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T15:37:57.144 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T15:37:57.145 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T15:37:57.146 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T15:37:57.147 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout:Install 150 Packages 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 24 Packages 2024-04-16T15:37:57.148 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:37:57.149 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 228 M 2024-04-16T15:37:57.149 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-04-16T15:37:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:(8/174): ceph-osd-19.0.0-2868.gc3c87ed0.el9.x86 17 MB/s | 17 MB 00:00 2024-04-16T15:37:57.332 INFO:teuthology.orchestra.run.smithi071.stdout:(9/174): ceph-selinux-19.0.0-2868.gc3c87ed0.el9 508 kB/s | 25 kB 00:00 2024-04-16T15:37:58.721 INFO:teuthology.orchestra.run.smithi122.stdout:(1/174): ceph-19.0.0-2868.gc3c87ed0.el9.x86_64. 9.2 kB/s | 6.6 kB 00:00 2024-04-16T15:37:58.991 INFO:teuthology.orchestra.run.smithi071.stdout:(10/174): ceph-common-19.0.0-2868.gc3c87ed0.el9 5.5 MB/s | 21 MB 00:03 2024-04-16T15:37:59.066 INFO:teuthology.orchestra.run.smithi071.stdout:(11/174): cephfs-mirror-19.0.0-2868.gc3c87ed0.e 2.9 MB/s | 225 kB 00:00 2024-04-16T15:37:59.158 INFO:teuthology.orchestra.run.smithi071.stdout:(12/174): cephfs-shell-19.0.0-2868.gc3c87ed0.el 296 kB/s | 27 kB 00:00 2024-04-16T15:37:59.275 INFO:teuthology.orchestra.run.smithi071.stdout:(13/174): ceph-radosgw-19.0.0-2868.gc3c87ed0.el 3.8 MB/s | 10 MB 00:02 2024-04-16T15:37:59.291 INFO:teuthology.orchestra.run.smithi071.stdout:(14/174): libcephfs-devel-19.0.0-2868.gc3c87ed0 237 kB/s | 32 kB 00:00 2024-04-16T15:37:59.400 INFO:teuthology.orchestra.run.smithi071.stdout:(15/174): libcephsqlite-19.0.0-2868.gc3c87ed0.e 1.5 MB/s | 163 kB 00:00 2024-04-16T15:37:59.484 INFO:teuthology.orchestra.run.smithi071.stdout:(16/174): libcephfs2-19.0.0-2868.gc3c87ed0.el9. 3.5 MB/s | 746 kB 00:00 2024-04-16T15:37:59.509 INFO:teuthology.orchestra.run.smithi071.stdout:(17/174): librados-devel-19.0.0-2868.gc3c87ed0. 1.1 MB/s | 127 kB 00:00 2024-04-16T15:37:59.570 INFO:teuthology.orchestra.run.smithi122.stdout:(2/174): ceph-fuse-19.0.0-2868.gc3c87ed0.el9.x8 1.0 MB/s | 890 kB 00:00 2024-04-16T15:37:59.684 INFO:teuthology.orchestra.run.smithi071.stdout:(18/174): libradosstriper1-19.0.0-2868.gc3c87ed 2.4 MB/s | 500 kB 00:00 2024-04-16T15:37:59.754 INFO:teuthology.orchestra.run.smithi122.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 790 kB/s | 145 kB 00:00 2024-04-16T15:37:59.776 INFO:teuthology.orchestra.run.smithi071.stdout:(19/174): python3-ceph-argparse-19.0.0-2868.gc3 491 kB/s | 45 kB 00:00 2024-04-16T15:37:59.876 INFO:teuthology.orchestra.run.smithi071.stdout:(20/174): python3-ceph-common-19.0.0-2868.gc3c8 1.3 MB/s | 135 kB 00:00 2024-04-16T15:37:59.985 INFO:teuthology.orchestra.run.smithi071.stdout:(21/174): python3-cephfs-19.0.0-2868.gc3c87ed0. 1.5 MB/s | 161 kB 00:00 2024-04-16T15:38:00.454 INFO:teuthology.orchestra.run.smithi122.stdout:(4/174): ceph-base-19.0.0-2868.gc3c87ed0.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-16T15:38:00.654 INFO:teuthology.orchestra.run.smithi122.stdout:(5/174): ceph-mgr-19.0.0-2868.gc3c87ed0.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-04-16T15:38:00.719 INFO:teuthology.orchestra.run.smithi071.stdout:(22/174): ceph-test-19.0.0-2868.gc3c87ed0.el9.x 14 MB/s | 49 MB 00:03 2024-04-16T15:38:00.744 INFO:teuthology.orchestra.run.smithi071.stdout:(23/174): python3-rados-19.0.0-2868.gc3c87ed0.e 422 kB/s | 320 kB 00:00 2024-04-16T15:38:00.769 INFO:teuthology.orchestra.run.smithi071.stdout:(24/174): python3-rbd-19.0.0-2868.gc3c87ed0.el9 5.8 MB/s | 298 kB 00:00 2024-04-16T15:38:00.853 INFO:teuthology.orchestra.run.smithi071.stdout:(25/174): librgw2-19.0.0-2868.gc3c87ed0.el9.x86 3.9 MB/s | 5.2 MB 00:01 2024-04-16T15:38:00.878 INFO:teuthology.orchestra.run.smithi071.stdout:(26/174): rbd-fuse-19.0.0-2868.gc3c87ed0.el9.x8 787 kB/s | 85 kB 00:00 2024-04-16T15:38:00.904 INFO:teuthology.orchestra.run.smithi071.stdout:(27/174): python3-rgw-19.0.0-2868.gc3c87ed0.el9 624 kB/s | 99 kB 00:00 2024-04-16T15:38:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:(28/174): rbd-nbd-19.0.0-2868.gc3c87ed0.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-16T15:38:00.946 INFO:teuthology.orchestra.run.smithi122.stdout:(6/174): ceph-mds-19.0.0-2868.gc3c87ed0.el9.x86 2.0 MB/s | 2.4 MB 00:01 2024-04-16T15:38:00.971 INFO:teuthology.orchestra.run.smithi071.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2868.g 513 kB/s | 34 kB 00:00 2024-04-16T15:38:01.012 INFO:teuthology.orchestra.run.smithi071.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2868.gc3c87ed 2.1 MB/s | 142 kB 00:00 2024-04-16T15:38:01.138 INFO:teuthology.orchestra.run.smithi122.stdout:(7/174): ceph-mon-19.0.0-2868.gc3c87ed0.el9.x86 9.7 MB/s | 4.7 MB 00:00 2024-04-16T15:38:01.429 INFO:teuthology.orchestra.run.smithi071.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2868.gc3c87 7.7 MB/s | 3.5 MB 00:00 2024-04-16T15:38:01.480 INFO:teuthology.orchestra.run.smithi071.stdout:(32/174): rbd-mirror-19.0.0-2868.gc3c87ed0.el9. 4.9 MB/s | 3.1 MB 00:00 2024-04-16T15:38:01.505 INFO:teuthology.orchestra.run.smithi071.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2868.gc3 3.2 MB/s | 249 kB 00:00 2024-04-16T15:38:01.555 INFO:teuthology.orchestra.run.smithi071.stdout:(34/174): ceph-mgr-rook-19.0.0-2868.gc3c87ed0.e 657 kB/s | 49 kB 00:00 2024-04-16T15:38:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2868.gc 219 kB/s | 16 kB 00:00 2024-04-16T15:38:01.630 INFO:teuthology.orchestra.run.smithi071.stdout:(36/174): ceph-volume-19.0.0-2868.gc3c87ed0.el9 3.4 MB/s | 264 kB 00:00 2024-04-16T15:38:01.723 INFO:teuthology.orchestra.run.smithi071.stdout:(37/174): cephfs-top-19.0.0-2868.gc3c87ed0.el9. 199 kB/s | 18 kB 00:00 2024-04-16T15:38:01.756 INFO:teuthology.orchestra.run.smithi071.stdout:(38/174): cephadm-19.0.0-2868.gc3c87ed0.el9.noa 4.2 MB/s | 762 kB 00:00 2024-04-16T15:38:01.915 INFO:teuthology.orchestra.run.smithi071.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-04-16T15:38:01.948 INFO:teuthology.orchestra.run.smithi071.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2024-04-16T15:38:01.974 INFO:teuthology.orchestra.run.smithi071.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-04-16T15:38:01.999 INFO:teuthology.orchestra.run.smithi071.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-16T15:38:02.024 INFO:teuthology.orchestra.run.smithi071.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2024-04-16T15:38:02.116 INFO:teuthology.orchestra.run.smithi071.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-16T15:38:02.133 INFO:teuthology.orchestra.run.smithi071.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-04-16T15:38:02.150 INFO:teuthology.orchestra.run.smithi071.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 300 kB/s | 10 kB 00:00 2024-04-16T15:38:02.175 INFO:teuthology.orchestra.run.smithi071.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-04-16T15:38:02.192 INFO:teuthology.orchestra.run.smithi071.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-04-16T15:38:02.213 INFO:teuthology.orchestra.run.smithi122.stdout:(8/174): ceph-radosgw-19.0.0-2868.gc3c87ed0.el9 9.3 MB/s | 10 MB 00:01 2024-04-16T15:38:02.226 INFO:teuthology.orchestra.run.smithi071.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-16T15:38:02.251 INFO:teuthology.orchestra.run.smithi071.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2024-04-16T15:38:02.272 INFO:teuthology.orchestra.run.smithi122.stdout:(9/174): ceph-selinux-19.0.0-2868.gc3c87ed0.el9 432 kB/s | 25 kB 00:00 2024-04-16T15:38:02.276 INFO:teuthology.orchestra.run.smithi071.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 46 kB 00:00 2024-04-16T15:38:02.302 INFO:teuthology.orchestra.run.smithi071.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-16T15:38:02.327 INFO:teuthology.orchestra.run.smithi071.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-16T15:38:02.353 INFO:teuthology.orchestra.run.smithi071.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-04-16T15:38:02.377 INFO:teuthology.orchestra.run.smithi071.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-04-16T15:38:02.503 INFO:teuthology.orchestra.run.smithi071.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-16T15:38:02.545 INFO:teuthology.orchestra.run.smithi071.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 9.3 MB/s | 1.5 MB 00:00 2024-04-16T15:38:02.579 INFO:teuthology.orchestra.run.smithi071.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 817 kB 00:00 2024-04-16T15:38:02.604 INFO:teuthology.orchestra.run.smithi071.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-04-16T15:38:02.672 INFO:teuthology.orchestra.run.smithi122.stdout:(10/174): ceph-common-19.0.0-2868.gc3c87ed0.el9 4.5 MB/s | 21 MB 00:04 2024-04-16T15:38:02.696 INFO:teuthology.orchestra.run.smithi071.stdout:(60/174): ceph-mgr-diskprediction-local-19.0.0- 4.4 MB/s | 7.4 MB 00:01 2024-04-16T15:38:02.789 INFO:teuthology.orchestra.run.smithi122.stdout:(11/174): cephfs-mirror-19.0.0-2868.gc3c87ed0.e 1.9 MB/s | 225 kB 00:00 2024-04-16T15:38:02.804 INFO:teuthology.orchestra.run.smithi071.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2024-04-16T15:38:02.838 INFO:teuthology.orchestra.run.smithi071.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.4 MB/s | 1.0 MB 00:00 2024-04-16T15:38:02.863 INFO:teuthology.orchestra.run.smithi071.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 472 kB/s | 78 kB 00:00 2024-04-16T15:38:02.873 INFO:teuthology.orchestra.run.smithi122.stdout:(12/174): cephfs-shell-19.0.0-2868.gc3c87ed0.el 325 kB/s | 27 kB 00:00 2024-04-16T15:38:02.965 INFO:teuthology.orchestra.run.smithi122.stdout:(13/174): libcephfs-devel-19.0.0-2868.gc3c87ed0 346 kB/s | 32 kB 00:00 2024-04-16T15:38:03.055 INFO:teuthology.orchestra.run.smithi071.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-04-16T15:38:03.080 INFO:teuthology.orchestra.run.smithi071.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 114 kB/s | 24 kB 00:00 2024-04-16T15:38:03.106 INFO:teuthology.orchestra.run.smithi071.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 76 kB/s | 18 kB 00:00 2024-04-16T15:38:03.123 INFO:teuthology.orchestra.run.smithi122.stdout:(14/174): libcephfs2-19.0.0-2868.gc3c87ed0.el9. 4.6 MB/s | 746 kB 00:00 2024-04-16T15:38:03.130 INFO:teuthology.orchestra.run.smithi071.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-16T15:38:03.156 INFO:teuthology.orchestra.run.smithi071.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 636 kB/s | 32 kB 00:00 2024-04-16T15:38:03.215 INFO:teuthology.orchestra.run.smithi122.stdout:(15/174): libcephsqlite-19.0.0-2868.gc3c87ed0.e 1.7 MB/s | 163 kB 00:00 2024-04-16T15:38:03.256 INFO:teuthology.orchestra.run.smithi071.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-04-16T15:38:03.282 INFO:teuthology.orchestra.run.smithi071.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-04-16T15:38:03.282 INFO:teuthology.orchestra.run.smithi122.stdout:(16/174): librados-devel-19.0.0-2868.gc3c87ed0. 1.9 MB/s | 127 kB 00:00 2024-04-16T15:38:03.307 INFO:teuthology.orchestra.run.smithi071.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-04-16T15:38:03.349 INFO:teuthology.orchestra.run.smithi071.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 752 kB/s | 69 kB 00:00 2024-04-16T15:38:03.374 INFO:teuthology.orchestra.run.smithi071.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 619 kB/s | 41 kB 00:00 2024-04-16T15:38:03.391 INFO:teuthology.orchestra.run.smithi122.stdout:(17/174): libradosstriper1-19.0.0-2868.gc3c87ed 4.5 MB/s | 500 kB 00:00 2024-04-16T15:38:03.399 INFO:teuthology.orchestra.run.smithi071.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.5 MB/s | 238 kB 00:00 2024-04-16T15:38:03.424 INFO:teuthology.orchestra.run.smithi071.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-16T15:38:03.449 INFO:teuthology.orchestra.run.smithi071.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-04-16T15:38:03.475 INFO:teuthology.orchestra.run.smithi071.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-16T15:38:03.500 INFO:teuthology.orchestra.run.smithi071.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 570 kB/s | 43 kB 00:00 2024-04-16T15:38:03.842 INFO:teuthology.orchestra.run.smithi071.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-04-16T15:38:03.867 INFO:teuthology.orchestra.run.smithi071.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 106 kB/s | 39 kB 00:00 2024-04-16T15:38:03.884 INFO:teuthology.orchestra.run.smithi071.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 107 kB/s | 41 kB 00:00 2024-04-16T15:38:03.918 INFO:teuthology.orchestra.run.smithi071.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-04-16T15:38:03.943 INFO:teuthology.orchestra.run.smithi071.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 598 kB/s | 35 kB 00:00 2024-04-16T15:38:03.968 INFO:teuthology.orchestra.run.smithi071.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 555 kB/s | 46 kB 00:00 2024-04-16T15:38:03.993 INFO:teuthology.orchestra.run.smithi071.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-16T15:38:04.010 INFO:teuthology.orchestra.run.smithi071.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-04-16T15:38:04.035 INFO:teuthology.orchestra.run.smithi071.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 943 kB/s | 63 kB 00:00 2024-04-16T15:38:04.061 INFO:teuthology.orchestra.run.smithi071.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 658 kB/s | 44 kB 00:00 2024-04-16T15:38:04.086 INFO:teuthology.orchestra.run.smithi071.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-16T15:38:04.108 INFO:teuthology.orchestra.run.smithi122.stdout:(18/174): librgw2-19.0.0-2868.gc3c87ed0.el9.x86 7.3 MB/s | 5.2 MB 00:00 2024-04-16T15:38:04.111 INFO:teuthology.orchestra.run.smithi071.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-16T15:38:04.153 INFO:teuthology.orchestra.run.smithi071.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-04-16T15:38:04.166 INFO:teuthology.orchestra.run.smithi122.stdout:(19/174): python3-ceph-argparse-19.0.0-2868.gc3 779 kB/s | 45 kB 00:00 2024-04-16T15:38:04.178 INFO:teuthology.orchestra.run.smithi071.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2024-04-16T15:38:04.212 INFO:teuthology.orchestra.run.smithi071.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-04-16T15:38:04.237 INFO:teuthology.orchestra.run.smithi071.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-04-16T15:38:04.242 INFO:teuthology.orchestra.run.smithi122.stdout:(20/174): python3-ceph-common-19.0.0-2868.gc3c8 1.8 MB/s | 135 kB 00:00 2024-04-16T15:38:04.308 INFO:teuthology.orchestra.run.smithi122.stdout:(21/174): python3-cephfs-19.0.0-2868.gc3c87ed0. 2.4 MB/s | 161 kB 00:00 2024-04-16T15:38:04.362 INFO:teuthology.orchestra.run.smithi071.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-16T15:38:04.375 INFO:teuthology.orchestra.run.smithi122.stdout:(22/174): python3-rados-19.0.0-2868.gc3c87ed0.e 4.7 MB/s | 320 kB 00:00 2024-04-16T15:38:04.387 INFO:teuthology.orchestra.run.smithi071.stdout:(96/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-04-16T15:38:04.404 INFO:teuthology.orchestra.run.smithi071.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-16T15:38:04.438 INFO:teuthology.orchestra.run.smithi071.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-04-16T15:38:04.451 INFO:teuthology.orchestra.run.smithi122.stdout:(23/174): python3-rbd-19.0.0-2868.gc3c87ed0.el9 3.9 MB/s | 298 kB 00:00 2024-04-16T15:38:04.505 INFO:teuthology.orchestra.run.smithi071.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-16T15:38:04.526 INFO:teuthology.orchestra.run.smithi122.stdout:(24/174): python3-rgw-19.0.0-2868.gc3c87ed0.el9 1.3 MB/s | 99 kB 00:00 2024-04-16T15:38:04.530 INFO:teuthology.orchestra.run.smithi071.stdout:(100/174): elfutils-libelf-devel-0.190-2.el9.x8 151 kB/s | 21 kB 00:00 2024-04-16T15:38:04.593 INFO:teuthology.orchestra.run.smithi122.stdout:(25/174): rbd-fuse-19.0.0-2868.gc3c87ed0.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-16T15:38:04.614 INFO:teuthology.orchestra.run.smithi071.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.2 MB/s | 245 kB 00:00 2024-04-16T15:38:04.656 INFO:teuthology.orchestra.run.smithi071.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.9 MB/s | 248 kB 00:00 2024-04-16T15:38:04.681 INFO:teuthology.orchestra.run.smithi071.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-04-16T15:38:04.706 INFO:teuthology.orchestra.run.smithi071.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-04-16T15:38:04.748 INFO:teuthology.orchestra.run.smithi071.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-16T15:38:04.849 INFO:teuthology.orchestra.run.smithi071.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 10 MB/s | 4.1 MB 00:00 2024-04-16T15:38:05.010 INFO:teuthology.orchestra.run.smithi122.stdout:(26/174): rbd-mirror-19.0.0-2868.gc3c87ed0.el9. 7.4 MB/s | 3.1 MB 00:00 2024-04-16T15:38:05.077 INFO:teuthology.orchestra.run.smithi122.stdout:(27/174): rbd-nbd-19.0.0-2868.gc3c87ed0.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-16T15:38:05.115 INFO:teuthology.orchestra.run.smithi071.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-04-16T15:38:05.144 INFO:teuthology.orchestra.run.smithi122.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2868.g 511 kB/s | 34 kB 00:00 2024-04-16T15:38:05.166 INFO:teuthology.orchestra.run.smithi071.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-16T15:38:05.203 INFO:teuthology.orchestra.run.smithi122.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2868.gc3c87ed 2.4 MB/s | 142 kB 00:00 2024-04-16T15:38:05.233 INFO:teuthology.orchestra.run.smithi071.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-04-16T15:38:05.261 INFO:teuthology.orchestra.run.smithi071.stdout:(110/174): gperftools-libs-2.9.1-2.el9.x86_64.r 603 kB/s | 309 kB 00:00 2024-04-16T15:38:05.308 INFO:teuthology.orchestra.run.smithi071.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-04-16T15:38:05.342 INFO:teuthology.orchestra.run.smithi071.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 836 kB/s | 67 kB 00:00 2024-04-16T15:38:05.384 INFO:teuthology.orchestra.run.smithi071.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-16T15:38:05.492 INFO:teuthology.orchestra.run.smithi071.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-16T15:38:05.526 INFO:teuthology.orchestra.run.smithi071.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 838 kB 00:00 2024-04-16T15:38:05.560 INFO:teuthology.orchestra.run.smithi071.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 650 kB/s | 43 kB 00:00 2024-04-16T15:38:05.618 INFO:teuthology.orchestra.run.smithi071.stdout:(117/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-04-16T15:38:05.661 INFO:teuthology.orchestra.run.smithi071.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-16T15:38:05.669 INFO:teuthology.orchestra.run.smithi122.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2868.gc3c87 7.6 MB/s | 3.5 MB 00:00 2024-04-16T15:38:05.685 INFO:teuthology.orchestra.run.smithi071.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 202 kB/s | 32 kB 00:00 2024-04-16T15:38:05.710 INFO:teuthology.orchestra.run.smithi071.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-16T15:38:05.736 INFO:teuthology.orchestra.run.smithi071.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-16T15:38:05.770 INFO:teuthology.orchestra.run.smithi071.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi071.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-04-16T15:38:05.820 INFO:teuthology.orchestra.run.smithi071.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-16T15:38:05.870 INFO:teuthology.orchestra.run.smithi071.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-16T15:38:05.896 INFO:teuthology.orchestra.run.smithi071.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-16T15:38:05.946 INFO:teuthology.orchestra.run.smithi071.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-16T15:38:05.963 INFO:teuthology.orchestra.run.smithi071.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-16T15:38:05.980 INFO:teuthology.orchestra.run.smithi071.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-04-16T15:38:06.005 INFO:teuthology.orchestra.run.smithi071.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-16T15:38:06.038 INFO:teuthology.orchestra.run.smithi071.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-16T15:38:06.089 INFO:teuthology.orchestra.run.smithi071.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-16T15:38:06.106 INFO:teuthology.orchestra.run.smithi071.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-04-16T15:38:06.131 INFO:teuthology.orchestra.run.smithi071.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 856 kB/s | 79 kB 00:00 2024-04-16T15:38:06.156 INFO:teuthology.orchestra.run.smithi071.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-04-16T15:38:06.190 INFO:teuthology.orchestra.run.smithi071.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-16T15:38:06.215 INFO:teuthology.orchestra.run.smithi071.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-16T15:38:06.232 INFO:teuthology.orchestra.run.smithi071.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-16T15:38:06.257 INFO:teuthology.orchestra.run.smithi071.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-16T15:38:06.283 INFO:teuthology.orchestra.run.smithi071.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-04-16T15:38:06.308 INFO:teuthology.orchestra.run.smithi071.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-16T15:38:06.341 INFO:teuthology.orchestra.run.smithi071.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-04-16T15:38:06.367 INFO:teuthology.orchestra.run.smithi071.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-04-16T15:38:06.392 INFO:teuthology.orchestra.run.smithi071.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-04-16T15:38:06.417 INFO:teuthology.orchestra.run.smithi071.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-16T15:38:06.451 INFO:teuthology.orchestra.run.smithi071.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-16T15:38:06.476 INFO:teuthology.orchestra.run.smithi071.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-16T15:38:06.501 INFO:teuthology.orchestra.run.smithi071.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 237 kB/s | 20 kB 00:00 2024-04-16T15:38:06.530 INFO:teuthology.orchestra.run.smithi071.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-16T15:38:06.577 INFO:teuthology.orchestra.run.smithi071.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-16T15:38:06.694 INFO:teuthology.orchestra.run.smithi071.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-16T15:38:06.761 INFO:teuthology.orchestra.run.smithi071.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-04-16T15:38:06.795 INFO:teuthology.orchestra.run.smithi071.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-16T15:38:06.836 INFO:teuthology.orchestra.run.smithi071.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.9 MB/s | 248 kB 00:00 2024-04-16T15:38:06.878 INFO:teuthology.orchestra.run.smithi071.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-16T15:38:06.919 INFO:teuthology.orchestra.run.smithi122.stdout:(31/174): ceph-test-19.0.0-2868.gc3c87ed0.el9.x 10 MB/s | 49 MB 00:04 2024-04-16T15:38:06.954 INFO:teuthology.orchestra.run.smithi071.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 952 kB/s | 71 kB 00:00 2024-04-16T15:38:06.987 INFO:teuthology.orchestra.run.smithi071.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 36 kB 00:00 2024-04-16T15:38:07.012 INFO:teuthology.orchestra.run.smithi122.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 5.5 MB/s | 7.4 MB 00:01 2024-04-16T15:38:07.037 INFO:teuthology.orchestra.run.smithi122.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2868.gc3 2.1 MB/s | 249 kB 00:00 2024-04-16T15:38:07.087 INFO:teuthology.orchestra.run.smithi071.stdout:(158/174): librbd1-19.0.0-2868.gc3c87ed0.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-16T15:38:07.113 INFO:teuthology.orchestra.run.smithi071.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-04-16T15:38:07.163 INFO:teuthology.orchestra.run.smithi071.stdout:(160/174): librados2-19.0.0-2868.gc3c87ed0.el9. 5.1 MB/s | 3.4 MB 00:00 2024-04-16T15:38:07.188 INFO:teuthology.orchestra.run.smithi071.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 306 kB/s | 23 kB 00:00 2024-04-16T15:38:07.204 INFO:teuthology.orchestra.run.smithi122.stdout:(34/174): ceph-osd-19.0.0-2868.gc3c87ed0.el9.x8 2.7 MB/s | 17 MB 00:06 2024-04-16T15:38:07.213 INFO:teuthology.orchestra.run.smithi071.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-04-16T15:38:07.229 INFO:teuthology.orchestra.run.smithi122.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2868.gc 86 kB/s | 16 kB 00:00 2024-04-16T15:38:07.238 INFO:teuthology.orchestra.run.smithi071.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-16T15:38:07.254 INFO:teuthology.orchestra.run.smithi122.stdout:(36/174): ceph-mgr-rook-19.0.0-2868.gc3c87ed0.e 203 kB/s | 49 kB 00:00 2024-04-16T15:38:07.264 INFO:teuthology.orchestra.run.smithi071.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-04-16T15:38:07.289 INFO:teuthology.orchestra.run.smithi071.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-04-16T15:38:07.313 INFO:teuthology.orchestra.run.smithi122.stdout:(37/174): cephfs-top-19.0.0-2868.gc3c87ed0.el9. 313 kB/s | 18 kB 00:00 2024-04-16T15:38:07.314 INFO:teuthology.orchestra.run.smithi071.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 846 kB/s | 64 kB 00:00 2024-04-16T15:38:07.338 INFO:teuthology.orchestra.run.smithi122.stdout:(38/174): ceph-volume-19.0.0-2868.gc3c87ed0.el9 1.9 MB/s | 264 kB 00:00 2024-04-16T15:38:07.339 INFO:teuthology.orchestra.run.smithi071.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 380 kB/s | 29 kB 00:00 2024-04-16T15:38:07.363 INFO:teuthology.orchestra.run.smithi122.stdout:(39/174): cephadm-19.0.0-2868.gc3c87ed0.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-16T15:38:07.415 INFO:teuthology.orchestra.run.smithi071.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-04-16T15:38:07.465 INFO:teuthology.orchestra.run.smithi071.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-04-16T15:38:07.491 INFO:teuthology.orchestra.run.smithi071.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.1 MB/s | 465 kB 00:00 2024-04-16T15:38:07.522 INFO:teuthology.orchestra.run.smithi122.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 983 kB/s | 106 kB 00:00 2024-04-16T15:38:07.524 INFO:teuthology.orchestra.run.smithi071.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 532 kB/s | 31 kB 00:00 2024-04-16T15:38:07.547 INFO:teuthology.orchestra.run.smithi122.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 662 kB 00:00 2024-04-16T15:38:07.566 INFO:teuthology.orchestra.run.smithi071.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-04-16T15:38:07.581 INFO:teuthology.orchestra.run.smithi122.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 920 kB 00:00 2024-04-16T15:38:07.606 INFO:teuthology.orchestra.run.smithi122.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-04-16T15:38:07.608 INFO:teuthology.orchestra.run.smithi071.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.5 MB/s | 2.3 MB 00:00 2024-04-16T15:38:07.633 INFO:teuthology.orchestra.run.smithi071.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 685 kB/s | 74 kB 00:00 2024-04-16T15:38:07.640 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-16T15:38:07.640 INFO:teuthology.orchestra.run.smithi071.stdout:Total 17 MB/s | 228 MB 00:13 2024-04-16T15:38:07.739 INFO:teuthology.orchestra.run.smithi122.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 165 kB/s | 32 kB 00:00 2024-04-16T15:38:07.757 INFO:teuthology.orchestra.run.smithi122.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 114 kB/s | 17 kB 00:00 2024-04-16T15:38:07.832 INFO:teuthology.orchestra.run.smithi122.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-16T15:38:07.849 INFO:teuthology.orchestra.run.smithi122.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 94 kB/s | 10 kB 00:00 2024-04-16T15:38:07.874 INFO:teuthology.orchestra.run.smithi122.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-04-16T15:38:07.900 INFO:teuthology.orchestra.run.smithi122.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-04-16T15:38:07.925 INFO:teuthology.orchestra.run.smithi122.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-04-16T15:38:07.950 INFO:teuthology.orchestra.run.smithi122.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-04-16T15:38:07.975 INFO:teuthology.orchestra.run.smithi122.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 624 kB/s | 46 kB 00:00 2024-04-16T15:38:08.009 INFO:teuthology.orchestra.run.smithi122.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-04-16T15:38:08.031 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-16T15:38:08.034 INFO:teuthology.orchestra.run.smithi122.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-04-16T15:38:08.059 INFO:teuthology.orchestra.run.smithi122.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-04-16T15:38:08.084 INFO:teuthology.orchestra.run.smithi122.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 457 kB/s | 34 kB 00:00 2024-04-16T15:38:08.210 INFO:teuthology.orchestra.run.smithi122.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-04-16T15:38:08.277 INFO:teuthology.orchestra.run.smithi122.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-04-16T15:38:08.304 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-16T15:38:08.304 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-16T15:38:08.335 INFO:teuthology.orchestra.run.smithi122.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 817 kB 00:00 2024-04-16T15:38:08.360 INFO:teuthology.orchestra.run.smithi122.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 894 kB/s | 132 kB 00:00 2024-04-16T15:38:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-04-16T15:38:08.502 INFO:teuthology.orchestra.run.smithi122.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.3 MB/s | 1.0 MB 00:00 2024-04-16T15:38:08.527 INFO:teuthology.orchestra.run.smithi122.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 475 kB/s | 78 kB 00:00 2024-04-16T15:38:08.628 INFO:teuthology.orchestra.run.smithi122.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 37 MB/s | 5.8 MB 00:00 2024-04-16T15:38:08.645 INFO:teuthology.orchestra.run.smithi122.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 215 kB/s | 24 kB 00:00 2024-04-16T15:38:08.662 INFO:teuthology.orchestra.run.smithi122.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 139 kB/s | 18 kB 00:00 2024-04-16T15:38:08.687 INFO:teuthology.orchestra.run.smithi122.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-04-16T15:38:08.712 INFO:teuthology.orchestra.run.smithi122.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 635 kB/s | 32 kB 00:00 2024-04-16T15:38:08.788 INFO:teuthology.orchestra.run.smithi122.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-04-16T15:38:08.813 INFO:teuthology.orchestra.run.smithi122.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-04-16T15:38:08.838 INFO:teuthology.orchestra.run.smithi122.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-04-16T15:38:08.863 INFO:teuthology.orchestra.run.smithi122.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 922 kB/s | 69 kB 00:00 2024-04-16T15:38:08.888 INFO:teuthology.orchestra.run.smithi122.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-04-16T15:38:08.913 INFO:teuthology.orchestra.run.smithi122.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-16T15:38:08.939 INFO:teuthology.orchestra.run.smithi122.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-16T15:38:08.964 INFO:teuthology.orchestra.run.smithi122.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-16T15:38:08.989 INFO:teuthology.orchestra.run.smithi122.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 630 kB/s | 47 kB 00:00 2024-04-16T15:38:09.014 INFO:teuthology.orchestra.run.smithi122.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 572 kB/s | 43 kB 00:00 2024-04-16T15:38:09.056 INFO:teuthology.orchestra.run.smithi122.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 581 kB/s | 39 kB 00:00 2024-04-16T15:38:09.132 INFO:teuthology.orchestra.run.smithi122.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 358 kB/s | 41 kB 00:00 2024-04-16T15:38:09.332 INFO:teuthology.orchestra.run.smithi122.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-04-16T15:38:09.365 INFO:teuthology.orchestra.run.smithi122.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 719 kB/s | 220 kB 00:00 2024-04-16T15:38:09.399 INFO:teuthology.orchestra.run.smithi122.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 131 kB/s | 35 kB 00:00 2024-04-16T15:38:09.424 INFO:teuthology.orchestra.run.smithi122.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 505 kB/s | 46 kB 00:00 2024-04-16T15:38:09.450 INFO:teuthology.orchestra.run.smithi122.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-04-16T15:38:09.475 INFO:teuthology.orchestra.run.smithi122.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 749 kB/s | 56 kB 00:00 2024-04-16T15:38:09.500 INFO:teuthology.orchestra.run.smithi122.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 837 kB/s | 63 kB 00:00 2024-04-16T15:38:09.525 INFO:teuthology.orchestra.run.smithi122.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-04-16T15:38:09.551 INFO:teuthology.orchestra.run.smithi122.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-16T15:38:09.584 INFO:teuthology.orchestra.run.smithi122.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-04-16T15:38:09.618 INFO:teuthology.orchestra.run.smithi122.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-04-16T15:38:09.643 INFO:teuthology.orchestra.run.smithi122.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-04-16T15:38:09.668 INFO:teuthology.orchestra.run.smithi122.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-16T15:38:09.693 INFO:teuthology.orchestra.run.smithi122.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-16T15:38:09.802 INFO:teuthology.orchestra.run.smithi122.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-04-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi122.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-04-16T15:38:09.852 INFO:teuthology.orchestra.run.smithi122.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-16T15:38:09.886 INFO:teuthology.orchestra.run.smithi122.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-16T15:38:09.911 INFO:teuthology.orchestra.run.smithi122.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 256 kB/s | 21 kB 00:00 2024-04-16T15:38:09.937 INFO:teuthology.orchestra.run.smithi122.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 304 kB/s | 25 kB 00:00 2024-04-16T15:38:09.987 INFO:teuthology.orchestra.run.smithi122.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 4.8 MB/s | 245 kB 00:00 2024-04-16T15:38:10.012 INFO:teuthology.orchestra.run.smithi122.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 3.2 MB/s | 248 kB 00:00 2024-04-16T15:38:10.029 INFO:teuthology.orchestra.run.smithi122.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-16T15:38:10.096 INFO:teuthology.orchestra.run.smithi122.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 20 MB/s | 4.1 MB 00:00 2024-04-16T15:38:10.121 INFO:teuthology.orchestra.run.smithi122.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-04-16T15:38:10.155 INFO:teuthology.orchestra.run.smithi122.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-04-16T15:38:10.328 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-16T15:38:10.329 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-16T15:38:10.613 INFO:teuthology.orchestra.run.smithi122.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-04-16T15:38:10.997 INFO:teuthology.orchestra.run.smithi122.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 353 kB/s | 309 kB 00:00 2024-04-16T15:38:11.106 INFO:teuthology.orchestra.run.smithi122.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-16T15:38:11.227 INFO:teuthology.orchestra.run.smithi122.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 400 kB/s | 49 kB 00:00 2024-04-16T15:38:11.306 INFO:teuthology.orchestra.run.smithi122.stdout:(111/174): abseil-cpp-20211102.0-3.el9.x86_64.r 453 kB/s | 548 kB 00:01 2024-04-16T15:38:11.406 INFO:teuthology.orchestra.run.smithi122.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 377 kB/s | 67 kB 00:00 2024-04-16T15:38:11.573 INFO:teuthology.orchestra.run.smithi122.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-04-16T15:38:12.315 INFO:teuthology.orchestra.run.smithi122.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 739 kB/s | 548 kB 00:00 2024-04-16T15:38:12.416 INFO:teuthology.orchestra.run.smithi122.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-04-16T15:38:12.507 INFO:teuthology.orchestra.run.smithi122.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 761 kB/s | 838 kB 00:01 2024-04-16T15:38:12.533 INFO:teuthology.orchestra.run.smithi122.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-04-16T15:38:12.608 INFO:teuthology.orchestra.run.smithi122.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-16T15:38:12.933 INFO:teuthology.orchestra.run.smithi122.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 429 kB/s | 172 kB 00:00 2024-04-16T15:38:13.150 INFO:teuthology.orchestra.run.smithi122.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 661 kB/s | 358 kB 00:00 2024-04-16T15:38:13.259 INFO:teuthology.orchestra.run.smithi122.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-04-16T15:38:13.376 INFO:teuthology.orchestra.run.smithi122.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 685 kB/s | 303 kB 00:00 2024-04-16T15:38:13.651 INFO:teuthology.orchestra.run.smithi122.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 517 kB/s | 203 kB 00:00 2024-04-16T15:38:13.960 INFO:teuthology.orchestra.run.smithi122.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 469 kB/s | 144 kB 00:00 2024-04-16T15:38:14.093 INFO:teuthology.orchestra.run.smithi122.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-04-16T15:38:14.202 INFO:teuthology.orchestra.run.smithi122.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-16T15:38:14.311 INFO:teuthology.orchestra.run.smithi122.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-16T15:38:14.419 INFO:teuthology.orchestra.run.smithi122.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-16T15:38:14.528 INFO:teuthology.orchestra.run.smithi122.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-04-16T15:38:16.003 INFO:teuthology.orchestra.run.smithi122.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 709 kB/s | 1.0 MB 00:01 2024-04-16T15:38:16.103 INFO:teuthology.orchestra.run.smithi122.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 765 kB/s | 2.0 MB 00:02 2024-04-16T15:38:16.128 INFO:teuthology.orchestra.run.smithi122.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-04-16T15:38:16.337 INFO:teuthology.orchestra.run.smithi122.stdout:(133/174): libarrow-9.0.0-10.el9.x86_64.rpm 790 kB/s | 4.4 MB 00:05 2024-04-16T15:38:16.362 INFO:teuthology.orchestra.run.smithi122.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 247 kB/s | 58 kB 00:00 2024-04-16T15:38:16.387 INFO:teuthology.orchestra.run.smithi122.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 277 kB/s | 79 kB 00:00 2024-04-16T15:38:16.480 INFO:teuthology.orchestra.run.smithi122.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-16T15:38:16.580 INFO:teuthology.orchestra.run.smithi122.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-04-16T15:38:16.605 INFO:teuthology.orchestra.run.smithi122.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 414 kB/s | 90 kB 00:00 2024-04-16T15:38:16.680 INFO:teuthology.orchestra.run.smithi122.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-16T15:38:16.806 INFO:teuthology.orchestra.run.smithi122.stdout:(140/174): python3-pecan-1.4.2-3.el9.noarch.rpm 582 kB/s | 272 kB 00:00 2024-04-16T15:38:16.873 INFO:teuthology.orchestra.run.smithi122.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 308 kB/s | 59 kB 00:00 2024-04-16T15:38:16.898 INFO:teuthology.orchestra.run.smithi122.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 645 kB/s | 188 kB 00:00 2024-04-16T15:38:16.923 INFO:teuthology.orchestra.run.smithi122.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-04-16T15:38:16.982 INFO:teuthology.orchestra.run.smithi122.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-04-16T15:38:17.107 INFO:teuthology.orchestra.run.smithi122.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 488 kB/s | 90 kB 00:00 2024-04-16T15:38:17.224 INFO:teuthology.orchestra.run.smithi122.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-16T15:38:17.258 INFO:teuthology.orchestra.run.smithi122.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 638 kB/s | 230 kB 00:00 2024-04-16T15:38:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-16T15:38:17.500 INFO:teuthology.orchestra.run.smithi122.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 696 kB/s | 192 kB 00:00 2024-04-16T15:38:17.595 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T15:38:17.616 INFO:teuthology.orchestra.run.smithi122.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 673 kB/s | 427 kB 00:00 2024-04-16T15:38:17.647 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T15:38:17.684 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T15:38:17.695 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T15:38:17.741 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T15:38:17.796 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T15:38:17.829 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T15:38:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T15:38:17.917 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T15:38:18.108 INFO:teuthology.orchestra.run.smithi122.stdout:(150/174): librbd1-19.0.0-2868.gc3c87ed0.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-04-16T15:38:18.217 INFO:teuthology.orchestra.run.smithi122.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 21 MB/s | 2.3 MB 00:00 2024-04-16T15:38:18.245 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T15:38:18.314 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 10/198 2024-04-16T15:38:18.334 INFO:teuthology.orchestra.run.smithi122.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-04-16T15:38:18.429 INFO:teuthology.orchestra.run.smithi122.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 430 kB/s | 41 kB 00:00 2024-04-16T15:38:18.436 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 10/198 2024-04-16T15:38:18.474 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 11/198 2024-04-16T15:38:18.485 INFO:teuthology.orchestra.run.smithi122.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.4 MB/s | 248 kB 00:00 2024-04-16T15:38:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-16T15:38:18.565 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 11/198 2024-04-16T15:38:18.598 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.0.0-2868.gc3c87ed0.el9.x86_64 12/198 2024-04-16T15:38:18.627 INFO:teuthology.orchestra.run.smithi122.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 779 kB/s | 71 kB 00:00 2024-04-16T15:38:18.636 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T15:38:18.694 INFO:teuthology.orchestra.run.smithi122.stdout:(157/174): librados2-19.0.0-2868.gc3c87ed0.el9. 2.8 MB/s | 3.4 MB 00:01 2024-04-16T15:38:18.719 INFO:teuthology.orchestra.run.smithi122.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 397 kB/s | 36 kB 00:00 2024-04-16T15:38:18.786 INFO:teuthology.orchestra.run.smithi122.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 256 kB 00:00 2024-04-16T15:38:18.811 INFO:teuthology.orchestra.run.smithi122.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-04-16T15:38:18.815 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T15:38:18.836 INFO:teuthology.orchestra.run.smithi122.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 459 kB/s | 23 kB 00:00 2024-04-16T15:38:18.862 INFO:teuthology.orchestra.run.smithi122.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-16T15:38:18.871 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T15:38:18.887 INFO:teuthology.orchestra.run.smithi122.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-16T15:38:18.891 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.0.0-2868.gc3c87ed0.el9.x86_64 16/198 2024-04-16T15:38:18.912 INFO:teuthology.orchestra.run.smithi122.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-16T15:38:18.937 INFO:teuthology.orchestra.run.smithi122.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-16T15:38:18.963 INFO:teuthology.orchestra.run.smithi122.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-04-16T15:38:18.993 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2868.gc3c87ed0.el9.x86_64 16/198 2024-04-16T15:38:19.017 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.0.0-2868.gc3c87ed0.el9.x86 17/198 2024-04-16T15:38:19.030 INFO:teuthology.orchestra.run.smithi122.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-04-16T15:38:19.082 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2868.gc3c87ed0.el9.x86 17/198 2024-04-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi122.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 15 MB/s | 2.2 MB 00:00 2024-04-16T15:38:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T15:38:19.152 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T15:38:19.155 INFO:teuthology.orchestra.run.smithi122.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 18 MB/s | 2.3 MB 00:00 2024-04-16T15:38:19.168 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T15:38:19.180 INFO:teuthology.orchestra.run.smithi122.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.1 MB/s | 465 kB 00:00 2024-04-16T15:38:19.206 INFO:teuthology.orchestra.run.smithi122.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 619 kB/s | 31 kB 00:00 2024-04-16T15:38:19.264 INFO:teuthology.orchestra.run.smithi122.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 74 kB 00:00 2024-04-16T15:38:19.306 INFO:teuthology.orchestra.run.smithi122.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-04-16T15:38:19.345 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T15:38:19.375 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T15:38:19.406 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T15:38:19.585 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T15:38:19.640 INFO:teuthology.orchestra.run.smithi122.stdout:(174/174): thrift-0.15.0-2.el9.x86_64.rpm 680 kB/s | 1.6 MB 00:02 2024-04-16T15:38:19.648 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-04-16T15:38:19.648 INFO:teuthology.orchestra.run.smithi122.stdout:Total 10 MB/s | 228 MB 00:22 2024-04-16T15:38:19.700 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T15:38:19.760 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T15:38:19.848 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T15:38:19.904 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T15:38:20.023 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-04-16T15:38:20.110 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T15:38:20.238 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T15:38:20.303 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-04-16T15:38:20.303 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-04-16T15:38:20.330 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T15:38:20.359 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T15:38:20.385 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T15:38:20.421 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T15:38:20.495 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T15:38:20.557 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T15:38:20.595 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T15:38:20.660 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T15:38:20.694 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T15:38:20.739 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T15:38:20.796 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T15:38:20.824 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T15:38:20.892 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T15:38:20.928 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.0.0-2868.gc3c87ed0.el9. 39/198 2024-04-16T15:38:20.964 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.0.0-2868.gc3c87ed0.el 40/198 2024-04-16T15:38:20.992 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.0.0-2868.gc3c87ed0.el9.x86_6 41/198 2024-04-16T15:38:21.060 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T15:38:21.106 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T15:38:21.180 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T15:38:21.313 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T15:38:21.366 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T15:38:21.883 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T15:38:21.962 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T15:38:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T15:38:22.091 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T15:38:22.101 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T15:38:22.158 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T15:38:22.213 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T15:38:22.251 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_6 53/198 2024-04-16T15:38:22.281 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T15:38:22.331 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-04-16T15:38:22.332 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-04-16T15:38:22.560 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T15:38:23.030 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T15:38:23.109 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T15:38:23.155 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T15:38:23.291 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T15:38:23.348 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T15:38:23.403 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T15:38:23.476 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T15:38:23.507 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T15:38:23.544 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T15:38:23.590 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T15:38:23.626 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T15:38:23.669 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T15:38:23.711 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T15:38:23.844 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T15:38:23.892 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T15:38:23.922 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T15:38:23.963 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T15:38:24.056 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T15:38:24.105 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T15:38:24.145 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T15:38:24.171 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T15:38:24.195 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T15:38:24.272 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T15:38:24.395 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T15:38:24.493 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T15:38:24.562 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T15:38:24.610 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T15:38:24.665 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T15:38:24.850 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T15:38:24.891 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T15:38:25.018 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T15:38:25.064 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T15:38:25.096 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T15:38:25.127 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T15:38:25.155 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T15:38:25.190 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T15:38:25.228 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T15:38:25.259 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T15:38:25.290 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T15:38:25.366 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T15:38:25.400 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T15:38:25.441 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T15:38:25.483 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T15:38:25.518 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T15:38:25.614 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T15:38:25.705 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T15:38:25.749 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T15:38:25.788 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T15:38:25.835 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T15:38:25.931 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T15:38:25.982 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T15:38:26.699 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T15:38:26.761 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T15:38:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T15:38:26.843 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T15:38:26.930 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T15:38:26.971 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T15:38:27.146 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T15:38:27.177 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T15:38:27.255 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T15:38:27.884 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T15:38:28.005 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T15:38:29.210 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T15:38:29.292 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T15:38:29.323 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T15:38:29.354 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T15:38:29.382 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T15:38:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-04-16T15:38:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T15:38:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T15:38:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T15:38:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T15:38:29.703 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T15:38:29.736 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 124/198 2024-04-16T15:38:29.752 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T15:38:29.805 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T15:38:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 124/198 2024-04-16T15:38:29.845 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T15:38:29.849 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 125/198 2024-04-16T15:38:29.871 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T15:38:29.886 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T15:38:29.927 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T15:38:30.250 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T15:38:30.322 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 10/198 2024-04-16T15:38:30.363 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T15:38:30.461 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 10/198 2024-04-16T15:38:30.490 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 11/198 2024-04-16T15:38:30.576 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 128/198 2024-04-16T15:38:30.583 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 11/198 2024-04-16T15:38:30.618 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.0.0-2868.gc3c87ed0.el9.x86_64 12/198 2024-04-16T15:38:30.668 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T15:38:30.836 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 128/198 2024-04-16T15:38:30.862 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T15:38:30.866 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 129/198 2024-04-16T15:38:30.913 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T15:38:30.941 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.0.0-2868.gc3c87ed0.el9.x86_64 16/198 2024-04-16T15:38:31.029 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2868.gc3c87ed0.el9.x86_64 16/198 2024-04-16T15:38:31.058 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.0.0-2868.gc3c87ed0.el9.x86 17/198 2024-04-16T15:38:31.106 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2868.gc3c87ed0.el9.x86 17/198 2024-04-16T15:38:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T15:38:31.168 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T15:38:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T15:38:31.408 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T15:38:31.448 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T15:38:31.635 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T15:38:31.783 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T15:38:31.826 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T15:38:31.921 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T15:38:31.970 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T15:38:32.168 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T15:38:32.275 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T15:38:32.369 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T15:38:32.409 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T15:38:32.435 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T15:38:32.479 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T15:38:32.552 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T15:38:32.606 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T15:38:32.645 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T15:38:32.709 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T15:38:32.744 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T15:38:32.787 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T15:38:32.821 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T15:38:32.858 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T15:38:32.929 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T15:38:32.963 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.0.0-2868.gc3c87ed0.el9. 39/198 2024-04-16T15:38:33.007 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.0.0-2868.gc3c87ed0.el 40/198 2024-04-16T15:38:33.035 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.0.0-2868.gc3c87ed0.el9.x86_6 41/198 2024-04-16T15:38:33.104 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T15:38:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 130/198 2024-04-16T15:38:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T15:38:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 130/198 2024-04-16T15:38:33.223 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T15:38:33.357 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T15:38:33.402 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T15:38:33.709 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 130/198 2024-04-16T15:38:33.748 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.0.0-2868.gc3c87ed0.el9.x86_64 131/198 2024-04-16T15:38:33.849 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.0.0-2868.gc3c87ed0.el9.x86_64 131/198 2024-04-16T15:38:33.849 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T15:38:33.850 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T15:38:33.850 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:33.900 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.0.0-2868.gc3c87ed0.el9.x86_64 132/198 2024-04-16T15:38:33.923 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T15:38:33.998 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T15:38:34.087 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T15:38:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T15:38:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T15:38:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T15:38:34.223 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T15:38:34.279 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_6 53/198 2024-04-16T15:38:34.316 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T15:38:34.594 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T15:38:34.773 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T15:38:34.860 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T15:38:34.904 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T15:38:35.048 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T15:38:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T15:38:35.145 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T15:38:35.218 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T15:38:35.250 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T15:38:35.287 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T15:38:35.333 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T15:38:35.369 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T15:38:35.412 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T15:38:35.454 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T15:38:35.589 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T15:38:35.643 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T15:38:35.673 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T15:38:35.715 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T15:38:35.809 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T15:38:35.848 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T15:38:35.888 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T15:38:35.913 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T15:38:35.938 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T15:38:36.018 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T15:38:36.147 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T15:38:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T15:38:36.288 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T15:38:36.337 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T15:38:36.393 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T15:38:36.585 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T15:38:36.626 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T15:38:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T15:38:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T15:38:36.856 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T15:38:36.895 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T15:38:36.924 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T15:38:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T15:38:36.997 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T15:38:37.028 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T15:38:37.058 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T15:38:37.134 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T15:38:37.168 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T15:38:37.202 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T15:38:37.243 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T15:38:37.278 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T15:38:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T15:38:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T15:38:37.492 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T15:38:37.524 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T15:38:37.571 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T15:38:37.669 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T15:38:37.725 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T15:38:38.445 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T15:38:38.489 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T15:38:38.522 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T15:38:38.571 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T15:38:38.659 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T15:38:38.699 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T15:38:38.877 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T15:38:38.904 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T15:38:38.982 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T15:38:39.640 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T15:38:39.754 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T15:38:40.976 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T15:38:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T15:38:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T15:38:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T15:38:41.155 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T15:38:41.466 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T15:38:41.494 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 124/198 2024-04-16T15:38:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 124/198 2024-04-16T15:38:41.623 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 125/198 2024-04-16T15:38:41.659 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T15:38:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T15:38:42.163 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 128/198 2024-04-16T15:38:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 128/198 2024-04-16T15:38:42.248 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 129/198 2024-04-16T15:38:44.429 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 130/198 2024-04-16T15:38:44.461 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 130/198 2024-04-16T15:38:45.014 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 130/198 2024-04-16T15:38:45.048 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.0.0-2868.gc3c87ed0.el9.x86_64 131/198 2024-04-16T15:38:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.0.0-2868.gc3c87ed0.el9.x86_64 131/198 2024-04-16T15:38:45.121 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T15:38:45.121 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T15:38:45.121 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:38:45.157 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.0.0-2868.gc3c87ed0.el9.x86_64 132/198 2024-04-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2868.gc3c87ed0.el9.x86_64 132/198 2024-04-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-04-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-04-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-04-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-04-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-04-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-04-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-04-16T15:38:45.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:45.527 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.0.0-2868.gc3c87ed0.el9.noarch 133/198 2024-04-16T15:38:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.0.0-2868.gc3c87ed0.el9.noarch 133/198 2024-04-16T15:38:45.563 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:45.563 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T15:38:45.563 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:47.038 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 134/198 2024-04-16T15:38:47.065 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 134/198 2024-04-16T15:38:47.066 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:47.066 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T15:38:47.066 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T15:38:47.066 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T15:38:47.066 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:47.289 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.0.0-2868.gc3c87ed0.el9.x86_64 135/198 2024-04-16T15:38:47.312 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.0.0-2868.gc3c87ed0.el9.x86_64 135/198 2024-04-16T15:38:47.313 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:47.313 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T15:38:47.313 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T15:38:47.313 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T15:38:47.313 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:47.756 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.0.0-2868.gc3c87ed0.el9.x86_64 136/198 2024-04-16T15:38:47.780 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.0.0-2868.gc3c87ed0.el9.x86_64 136/198 2024-04-16T15:38:47.780 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:47.780 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T15:38:47.780 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T15:38:47.781 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T15:38:47.781 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:47.874 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T15:38:48.346 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 138/198 2024-04-16T15:38:48.373 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 138/198 2024-04-16T15:38:48.405 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2868.gc3c87ed0.e 139/198 2024-04-16T15:38:48.466 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2868.gc3c87ed0. 140/198 2024-04-16T15:38:48.495 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2868.gc3c87ed0.el9.noa 141/198 2024-04-16T15:38:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2868.gc3c87ed0.el9.noa 141/198 2024-04-16T15:38:49.564 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2868.gc3c87ed0.el9.n 142/198 2024-04-16T15:38:50.528 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2868.gc3c87ed0.el9.n 142/198 2024-04-16T15:38:50.556 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2868.gc3c 143/198 2024-04-16T15:38:50.657 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2868.gc3c 143/198 2024-04-16T15:38:50.820 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2868.gc3c87ed0.el 144/198 2024-04-16T15:38:50.850 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.0.0-2868.gc3c87ed0.el9.x86_64 145/198 2024-04-16T15:38:50.874 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2868.gc3c87ed0.el9.x86_64 145/198 2024-04-16T15:38:50.874 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:50.874 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T15:38:50.874 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T15:38:50.874 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T15:38:50.874 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:50.917 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.0.0-2868.gc3c87ed0.el9.noarch 146/198 2024-04-16T15:38:50.929 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2868.gc3c87ed0.el9.noarch 146/198 2024-04-16T15:38:50.978 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.0.0-2868.gc3c87ed0.el9.x86_64 147/198 2024-04-16T15:38:51.865 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T15:38:51.894 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 149/198 2024-04-16T15:38:51.918 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 149/198 2024-04-16T15:38:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T15:38:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T15:38:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T15:38:51.918 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:51.961 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2868.gc3c87 150/198 2024-04-16T15:38:51.984 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2868.gc3c87 150/198 2024-04-16T15:38:51.985 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:51.985 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T15:38:51.985 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:52.029 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 151/198 2024-04-16T15:38:52.052 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 151/198 2024-04-16T15:38:52.052 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:52.052 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T15:38:52.052 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T15:38:52.052 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T15:38:52.052 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:52.362 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 152/198 2024-04-16T15:38:52.386 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 152/198 2024-04-16T15:38:52.386 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:52.386 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T15:38:52.386 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T15:38:52.386 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T15:38:52.386 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:38:56.941 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.0.0-2868.gc3c87ed0.el9.x86_64 153/198 2024-04-16T15:38:56.983 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 154/198 2024-04-16T15:38:57.013 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 155/198 2024-04-16T15:38:57.033 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2868.gc3c87ed0.el9.x86_64 132/198 2024-04-16T15:38:57.033 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-04-16T15:38:57.033 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-04-16T15:38:57.033 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-04-16T15:38:57.033 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-04-16T15:38:57.033 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-04-16T15:38:57.033 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-04-16T15:38:57.033 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-04-16T15:38:57.034 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:38:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T15:38:57.187 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.0.0-2868.gc3c87ed0.el9.noarch 133/198 2024-04-16T15:38:57.203 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.0.0-2868.gc3c87ed0.el9.noarch 133/198 2024-04-16T15:38:57.203 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:57.203 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T15:38:57.203 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:38:57.221 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-shell-2:19.0.0-2868.gc3c87ed0.el9.x86_64 157/198 2024-04-16T15:38:57.331 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T15:38:57.379 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 159/198 2024-04-16T15:38:57.432 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T15:38:57.497 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T15:38:57.539 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T15:38:57.563 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_ 163/198 2024-04-16T15:38:57.604 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T15:38:57.634 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T15:38:57.687 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T15:38:57.756 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T15:38:57.790 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T15:38:58.371 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T15:38:58.413 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T15:38:58.433 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T15:38:58.683 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 134/198 2024-04-16T15:38:58.710 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 134/198 2024-04-16T15:38:58.710 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:58.710 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T15:38:58.710 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T15:38:58.710 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T15:38:58.711 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:38:58.942 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.0.0-2868.gc3c87ed0.el9.x86_64 135/198 2024-04-16T15:38:58.965 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.0.0-2868.gc3c87ed0.el9.x86_64 135/198 2024-04-16T15:38:58.966 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:58.966 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T15:38:58.966 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T15:38:58.966 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T15:38:58.966 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:38:59.247 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:19.0.0-2868.gc3c87ed0.el9.noarch 170/198 2024-04-16T15:38:59.324 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T15:38:59.366 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T15:38:59.401 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.0.0-2868.gc3c87ed0.el9.x86_64 136/198 2024-04-16T15:38:59.424 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.0.0-2868.gc3c87ed0.el9.x86_64 136/198 2024-04-16T15:38:59.424 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:38:59.424 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T15:38:59.424 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T15:38:59.424 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T15:38:59.424 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:38:59.462 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T15:38:59.494 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T15:38:59.511 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T15:38:59.519 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T15:38:59.545 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T15:38:59.571 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T15:38:59.571 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T15:38:59.659 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T15:38:59.697 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T15:38:59.697 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T15:38:59.781 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T15:38:59.809 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T15:38:59.841 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T15:38:59.866 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T15:38:59.891 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T15:38:59.928 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T15:38:59.956 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 138/198 2024-04-16T15:38:59.965 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T15:38:59.984 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 138/198 2024-04-16T15:38:59.996 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T15:39:00.017 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2868.gc3c87ed0.e 139/198 2024-04-16T15:39:00.021 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T15:39:00.046 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T15:39:00.071 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T15:39:00.080 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2868.gc3c87ed0. 140/198 2024-04-16T15:39:00.097 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T15:39:00.104 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T15:39:00.104 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T15:39:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2868.gc3c87ed0.el9.noa 141/198 2024-04-16T15:39:00.283 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T15:39:00.283 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T15:39:00.366 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T15:39:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T15:39:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T15:39:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T15:39:00.484 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T15:39:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T15:39:01.145 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2868.gc3c87ed0.el9.noa 141/198 2024-04-16T15:39:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2868.gc3c87ed0.el9.n 142/198 2024-04-16T15:39:02.148 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2868.gc3c87ed0.el9.n 142/198 2024-04-16T15:39:02.177 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2868.gc3c 143/198 2024-04-16T15:39:02.278 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2868.gc3c 143/198 2024-04-16T15:39:02.440 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2868.gc3c87ed0.el 144/198 2024-04-16T15:39:02.470 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.0.0-2868.gc3c87ed0.el9.x86_64 145/198 2024-04-16T15:39:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2868.gc3c87ed0.el9.x86_64 145/198 2024-04-16T15:39:02.495 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:39:02.495 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T15:39:02.495 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T15:39:02.495 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T15:39:02.495 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:02.546 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.0.0-2868.gc3c87ed0.el9.noarch 146/198 2024-04-16T15:39:02.557 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2868.gc3c87ed0.el9.noarch 146/198 2024-04-16T15:39:02.611 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.0.0-2868.gc3c87ed0.el9.x86_64 147/198 2024-04-16T15:39:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T15:39:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 149/198 2024-04-16T15:39:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 149/198 2024-04-16T15:39:03.546 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:39:03.547 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T15:39:03.547 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T15:39:03.547 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T15:39:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2868.gc3c87 150/198 2024-04-16T15:39:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2868.gc3c87 150/198 2024-04-16T15:39:03.614 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:39:03.614 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T15:39:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 151/198 2024-04-16T15:39:03.681 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 151/198 2024-04-16T15:39:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:39:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T15:39:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T15:39:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T15:39:03.681 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:03.992 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 152/198 2024-04-16T15:39:04.016 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 152/198 2024-04-16T15:39:04.016 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T15:39:04.016 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T15:39:04.016 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T15:39:04.016 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T15:39:04.016 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:06.049 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.0.0-2868.gc3c87ed0.el9.x86_64 1/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 3/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 4/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2868.gc3c87 5/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.0.0-2868.gc3c87ed0.el9.x86_64 6/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.0.0-2868.gc3c87ed0.el9.x86_64 7/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.0.0-2868.gc3c87ed0.el9.x86_64 8/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 9/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 10/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.0.0-2868.gc3c87ed0.el9.x86_64 11/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.0.0-2868.gc3c87ed0.el9.x86_64 12/198 2024-04-16T15:39:06.050 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 13/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-shell-2:19.0.0-2868.gc3c87ed0.el9.x86_64 14/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_ 15/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 16/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.0.0-2868.gc3c87ed0.el9.x86_64 17/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_6 18/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.0.0-2868.gc3c87ed0.el9.x86 19/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 20/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2868.gc3c87ed0.el 21/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.0.0-2868.gc3c87ed0.el9. 22/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.0.0-2868.gc3c87ed0.el9.x86_6 23/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.0.0-2868.gc3c87ed0.el9.x86_64 24/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 25/198 2024-04-16T15:39:06.051 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 26/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 27/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 28/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 29/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2868.gc3c87ed0. 30/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2868.gc3c87ed0.el9.noa 31/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2868.gc3c87ed0.el9.n 32/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2868.gc3c 33/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2868.gc3c87ed0.el 34/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2868.gc3c87ed0.el9.noarch 35/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2868.gc3c87ed0.e 36/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.0.0-2868.gc3c87ed0.el9.noarch 37/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 38/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:19.0.0-2868.gc3c87ed0.el9.noarch 39/198 2024-04-16T15:39:06.052 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T15:39:06.053 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T15:39:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T15:39:06.055 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T15:39:06.056 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T15:39:06.057 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T15:39:06.058 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T15:39:06.059 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T15:39:06.060 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 151/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 153/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T15:39:06.061 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T15:39:06.062 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T15:39:06.063 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T15:39:06.064 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T15:39:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.0.0-2868.gc3c87ed0.el9.x86_64 153/198 2024-04-16T15:39:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 154/198 2024-04-16T15:39:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 155/198 2024-04-16T15:39:08.695 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T15:39:08.876 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.0.0-2868.gc3c87ed0.el9.x86_64 157/198 2024-04-16T15:39:08.995 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T15:39:09.035 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 159/198 2024-04-16T15:39:09.096 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T15:39:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T15:39:09.149 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T15:39:09.149 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:09.149 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-04-16T15:39:09.149 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T15:39:09.149 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T15:39:09.149 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.150 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T15:39:09.151 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.152 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T15:39:09.153 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: jq-1.6-16.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T15:39:09.154 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T15:39:09.155 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T15:39:09.156 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T15:39:09.157 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T15:39:09.158 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T15:39:09.159 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T15:39:09.160 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T15:39:09.161 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T15:39:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T15:39:09.163 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T15:39:09.163 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:09.163 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-16T15:39:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T15:39:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_ 163/198 2024-04-16T15:39:09.260 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T15:39:09.289 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T15:39:09.353 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T15:39:09.418 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T15:39:09.455 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T15:39:09.630 DEBUG:teuthology.parallel:result is None 2024-04-16T15:39:09.977 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T15:39:10.011 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T15:39:10.031 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T15:39:10.858 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.0.0-2868.gc3c87ed0.el9.noarch 170/198 2024-04-16T15:39:10.940 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T15:39:10.981 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T15:39:11.063 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T15:39:11.096 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T15:39:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T15:39:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T15:39:11.206 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T15:39:11.206 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T15:39:11.293 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T15:39:11.323 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T15:39:11.323 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T15:39:11.408 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T15:39:11.440 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T15:39:11.475 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T15:39:11.501 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T15:39:11.526 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T15:39:11.562 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T15:39:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T15:39:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T15:39:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T15:39:11.698 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T15:39:11.723 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T15:39:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T15:39:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T15:39:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T15:39:11.943 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T15:39:11.943 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T15:39:12.051 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T15:39:12.082 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T15:39:12.109 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T15:39:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T15:39:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T15:39:12.226 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T15:39:17.910 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T15:39:17.910 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.0.0-2868.gc3c87ed0.el9.x86_64 1/198 2024-04-16T15:39:17.910 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2/198 2024-04-16T15:39:17.910 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 3/198 2024-04-16T15:39:17.910 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 4/198 2024-04-16T15:39:17.910 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2868.gc3c87 5/198 2024-04-16T15:39:17.910 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.0.0-2868.gc3c87ed0.el9.x86_64 6/198 2024-04-16T15:39:17.910 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.0.0-2868.gc3c87ed0.el9.x86_64 7/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.0.0-2868.gc3c87ed0.el9.x86_64 8/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 9/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 10/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.0.0-2868.gc3c87ed0.el9.x86_64 11/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.0.0-2868.gc3c87ed0.el9.x86_64 12/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 13/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.0.0-2868.gc3c87ed0.el9.x86_64 14/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_ 15/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 16/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.0.0-2868.gc3c87ed0.el9.x86_64 17/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_6 18/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.0.0-2868.gc3c87ed0.el9.x86 19/198 2024-04-16T15:39:17.911 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 20/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2868.gc3c87ed0.el 21/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.0.0-2868.gc3c87ed0.el9. 22/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.0.0-2868.gc3c87ed0.el9.x86_6 23/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.0.0-2868.gc3c87ed0.el9.x86_64 24/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 25/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 26/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 27/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 28/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 29/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2868.gc3c87ed0. 30/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2868.gc3c87ed0.el9.noa 31/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2868.gc3c87ed0.el9.n 32/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2868.gc3c 33/198 2024-04-16T15:39:17.912 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2868.gc3c87ed0.el 34/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2868.gc3c87ed0.el9.noarch 35/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2868.gc3c87ed0.e 36/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.0.0-2868.gc3c87ed0.el9.noarch 37/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 38/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.0.0-2868.gc3c87ed0.el9.noarch 39/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T15:39:17.913 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T15:39:17.914 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T15:39:17.915 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T15:39:17.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T15:39:17.917 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T15:39:17.918 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T15:39:17.919 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T15:39:17.920 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T15:39:17.921 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 151/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 153/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T15:39:17.922 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T15:39:17.923 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T15:39:17.924 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T15:39:17.925 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T15:39:17.926 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T15:39:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.0.0-2868.gc3c87ed0.el9.noarch 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: jq-1.6-16.el9.x86_64 2024-04-16T15:39:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T15:39:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T15:39:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T15:39:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T15:39:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T15:39:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T15:39:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.0.0-2868.gc3c87ed0.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T15:39:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T15:39:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T15:39:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:21.077 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-04-16T15:39:21.562 DEBUG:teuthology.parallel:result is None 2024-04-16T15:39:21.563 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f 2024-04-16T15:39:21.708 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T15:39:21.735 INFO:teuthology.orchestra.run.smithi071.stdout:19.0.0-2868.gc3c87ed0.el9 2024-04-16T15:39:21.735 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2868.gc3c87ed0.el9 2024-04-16T15:39:21.736 INFO:teuthology.task.install:The correct ceph version 19.0.0-2868.gc3c87ed0 is installed. 2024-04-16T15:39:21.737 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f 2024-04-16T15:39:21.959 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T15:39:21.983 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-2868.gc3c87ed0.el9 2024-04-16T15:39:21.983 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2868.gc3c87ed0.el9 2024-04-16T15:39:21.983 INFO:teuthology.task.install:The correct ceph version 19.0.0-2868.gc3c87ed0 is installed. 2024-04-16T15:39:21.985 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-16T15:39:21.985 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:21.985 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T15:39:22.022 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:22.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T15:39:22.056 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-16T15:39:22.057 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:22.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T15:39:22.093 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T15:39:22.163 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:22.163 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T15:39:22.192 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T15:39:22.261 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-16T15:39:22.261 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:22.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T15:39:22.310 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T15:39:22.380 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:22.380 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T15:39:22.408 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T15:39:22.477 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-16T15:39:22.477 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:22.477 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T15:39:22.506 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T15:39:22.574 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:22.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T15:39:22.602 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T15:39:22.671 INFO:teuthology.run_tasks:Running task ceph... 2024-04-16T15:39:22.750 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-16T15:39:22.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /var/log/ceph 2024-04-16T15:39:22.752 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-04-16T15:39:22.780 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-16T15:39:22.780 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-16T15:39:22.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-16T15:39:22.848 INFO:tasks.ceph:Creating extra log directories... 2024-04-16T15:39:22.848 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-16T15:39:22.890 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-16T15:39:22.924 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-16T15:39:22.924 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-16T15:39:22.924 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-16T15:39:22.924 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:22.924 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-16T15:39:22.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-16T15:39:23.094 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-16T15:39:23.122 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:23.122 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-16T15:39:23.138 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-16T15:39:23.265 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-16T15:39:23.299 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-16T15:39:23.300 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds debug zombie log segments': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-16T15:39:23.300 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189', 'branch': 'wip-lusov-shared-log-segment', 'description': 'fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover}', 'email': None, 'first_in_suite': False, 'job_id': '7658189', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-shared-log-segment'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds debug zombie log segments': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'ceph-fuse': {'disabled': True}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-shared-log-segment', 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 150, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 2402, 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f', 'sleep_before_teardown': 0, 'subset': '111/111111', 'suite': 'fs', 'suite_branch': 'wip-lusov-shared-log-segment', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi071.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEaLmnEMWE3/rOPSlclcAOSvPglNANZHh+V2bulxFaEAE5+8jQnVt2k/Pp0+fCPEMyOyYW5z+k5SEF+qNhZsT3s=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATWxCMAJge6GKTvl/K5DiJMRc7inzSVqkifT11tLL0Jml0L4eXlHil1S18hIMOzySdalstAjYssYekXInVLTrU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds debug zombie log segments': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'sha1': 'c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'exec': {'mon.a': ['ceph config set mgr mgr/crash/warn_recent_interval 0']}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_failover']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-16_12:36:42', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-16T15:39:23.300 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-16T15:39:23.302 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-16T15:39:23.350 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T15:39:23.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-04-16T15:39:23.419 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:23.419 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T15:39:23.443 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T15:39:23.444 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-04-16T15:39:23.498 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T15:39:23.499 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T15:39:23.499 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-16T15:39:23.499 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T15:39:23.499 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T15:39:23.499 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-16 15:39:00.905772508 +0000 2024-04-16T15:39:23.499 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-16 15:37:07.589604919 +0000 2024-04-16T15:39:23.499 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-16 15:37:07.589604919 +0000 2024-04-16T15:39:23.499 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-16 15:37:07.589604919 +0000 2024-04-16T15:39:23.499 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T15:39:23.566 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-16T15:39:23.567 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-16T15:39:23.567 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000277657 s, 1.8 MB/s 2024-04-16T15:39:23.568 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T15:39:23.625 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-04-16T15:39:23.679 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T15:39:23.680 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T15:39:23.680 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-16T15:39:23.680 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T15:39:23.680 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T15:39:23.680 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-16 15:39:00.905772508 +0000 2024-04-16T15:39:23.680 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-16 15:37:07.948598400 +0000 2024-04-16T15:39:23.680 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-16 15:37:07.948598400 +0000 2024-04-16T15:39:23.680 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-16 15:37:07.948598400 +0000 2024-04-16T15:39:23.680 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T15:39:23.747 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-16T15:39:23.747 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-16T15:39:23.747 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000169544 s, 3.0 MB/s 2024-04-16T15:39:23.748 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T15:39:23.805 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-04-16T15:39:23.859 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T15:39:23.860 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T15:39:23.860 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-16T15:39:23.860 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T15:39:23.860 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T15:39:23.860 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-16 15:39:00.905772508 +0000 2024-04-16T15:39:23.860 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-16 15:37:08.284592297 +0000 2024-04-16T15:39:23.860 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-16 15:37:08.284592297 +0000 2024-04-16T15:39:23.860 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-16 15:37:08.284592297 +0000 2024-04-16T15:39:23.860 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T15:39:23.926 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-16T15:39:23.926 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-16T15:39:23.926 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000191065 s, 2.7 MB/s 2024-04-16T15:39:23.927 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T15:39:23.984 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-04-16T15:39:24.039 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T15:39:24.039 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T15:39:24.039 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-16T15:39:24.039 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T15:39:24.039 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T15:39:24.040 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-16 15:39:00.906772489 +0000 2024-04-16T15:39:24.040 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-16 15:37:08.612586340 +0000 2024-04-16T15:39:24.040 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-16 15:37:08.612586340 +0000 2024-04-16T15:39:24.040 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-16 15:37:08.612586340 +0000 2024-04-16T15:39:24.040 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T15:39:24.106 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-16T15:39:24.106 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-16T15:39:24.106 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000187527 s, 2.7 MB/s 2024-04-16T15:39:24.107 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T15:39:24.165 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:24.165 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:24.165 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T15:39:24.185 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T15:39:24.186 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-04-16T15:39:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T15:39:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T15:39:24.244 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-16T15:39:24.244 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T15:39:24.244 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T15:39:24.244 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-16 15:39:12.586079740 +0000 2024-04-16T15:39:24.244 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-16 15:37:07.592825074 +0000 2024-04-16T15:39:24.244 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-16 15:37:07.592825074 +0000 2024-04-16T15:39:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-16 15:37:07.592825074 +0000 2024-04-16T15:39:24.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T15:39:24.314 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-16T15:39:24.314 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-16T15:39:24.314 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000296079 s, 1.7 MB/s 2024-04-16T15:39:24.316 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T15:39:24.374 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-04-16T15:39:24.431 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T15:39:24.432 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T15:39:24.432 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-16T15:39:24.432 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T15:39:24.432 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T15:39:24.432 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-16 15:39:12.586079740 +0000 2024-04-16T15:39:24.432 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-16 15:37:07.937816579 +0000 2024-04-16T15:39:24.433 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-16 15:37:07.937816579 +0000 2024-04-16T15:39:24.433 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-16 15:37:07.937816579 +0000 2024-04-16T15:39:24.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T15:39:24.501 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-16T15:39:24.501 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-16T15:39:24.502 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000189754 s, 2.7 MB/s 2024-04-16T15:39:24.504 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T15:39:24.561 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-04-16T15:39:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T15:39:24.621 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T15:39:24.622 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-16T15:39:24.622 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T15:39:24.622 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T15:39:24.622 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-16 15:39:12.587079716 +0000 2024-04-16T15:39:24.622 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-16 15:37:08.278808182 +0000 2024-04-16T15:39:24.622 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-16 15:37:08.278808182 +0000 2024-04-16T15:39:24.622 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-16 15:37:08.278808182 +0000 2024-04-16T15:39:24.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T15:39:24.692 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-16T15:39:24.692 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-16T15:39:24.692 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000169534 s, 3.0 MB/s 2024-04-16T15:39:24.693 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T15:39:24.751 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-16 15:39:12.587079716 +0000 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-16 15:37:08.628799564 +0000 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-16 15:37:08.628799564 +0000 2024-04-16T15:39:24.807 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-16 15:37:08.628799564 +0000 2024-04-16T15:39:24.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T15:39:24.874 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-16T15:39:24.874 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-16T15:39:24.879 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000217925 s, 2.3 MB/s 2024-04-16T15:39:24.880 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T15:39:24.937 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:24.937 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-16T15:39:24.937 INFO:tasks.ceph:Generating config... 2024-04-16T15:39:24.939 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-16T15:39:24.939 INFO:tasks.ceph:[client] debug client = 20 2024-04-16T15:39:24.939 INFO:tasks.ceph:[client] debug ms = 1 2024-04-16T15:39:24.939 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-16T15:39:24.939 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-16T15:39:24.939 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-16T15:39:24.939 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-16T15:39:24.939 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-16T15:39:24.939 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-16T15:39:24.939 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-16T15:39:24.939 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-16T15:39:24.939 INFO:tasks.ceph:[mds] mds debug zombie log segments = True 2024-04-16T15:39:24.939 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-16T15:39:24.939 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-16T15:39:24.940 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-16T15:39:24.940 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-16T15:39:24.940 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-16T15:39:24.940 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-16T15:39:24.941 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-16T15:39:24.941 INFO:tasks.ceph:Setting up mon.a... 2024-04-16T15:39:24.941 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-16T15:39:25.015 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.keyring 2024-04-16T15:39:25.017 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-16T15:39:25.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-16T15:39:25.097 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.71'), ('mon.b', '172.21.15.122'), ('mon.c', '[v2:172.21.15.122:3301,v1:172.21.15.122:6790]')] 2024-04-16T15:39:25.098 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.71,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds debug zombie log segments': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-16T15:39:25.100 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:25.100 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-16T15:39:25.154 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.71 --add b 172.21.15.122 --addv c '[v2:172.21.15.122:3301,v1:172.21.15.122:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:25.241 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:25.241 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: generated fsid c5697e20-5e32-4f13-b48e-2f7b8429c0b4 2024-04-16T15:39:25.241 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = quincy 2024-04-16T15:39:25.241 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 0 2024-04-16T15:39:25.241 INFO:teuthology.orchestra.run.smithi071.stdout:fsid c5697e20-5e32-4f13-b48e-2f7b8429c0b4 2024-04-16T15:39:25.241 INFO:teuthology.orchestra.run.smithi071.stdout:last_changed 2024-04-16T15:39:25.243629+0000 2024-04-16T15:39:25.242 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-16T15:39:25.243629+0000 2024-04-16T15:39:25.242 INFO:teuthology.orchestra.run.smithi071.stdout:min_mon_release 17 (quincy) 2024-04-16T15:39:25.242 INFO:teuthology.orchestra.run.smithi071.stdout:election_strategy: 1 2024-04-16T15:39:25.242 INFO:teuthology.orchestra.run.smithi071.stdout:0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-04-16T15:39:25.242 INFO:teuthology.orchestra.run.smithi071.stdout:1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.b 2024-04-16T15:39:25.242 INFO:teuthology.orchestra.run.smithi071.stdout:2: [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] mon.c 2024-04-16T15:39:25.242 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-16T15:39:25.243 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-16T15:39:25.258 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c5697e20-5e32-4f13-b48e-2f7b8429c0b4... 2024-04-16T15:39:25.259 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-16T15:39:25.301 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-16T15:39:25.351 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-04-16T15:39:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: chdir = "" 2024-04-16T15:39:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T15:39:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: auth supported = cephx 2024-04-16T15:39:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: filestore xattr use omap = true 2024-04-16T15:39:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: mon clock drift allowed = 1.000 2024-04-16T15:39:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: osd crush chooseleaf type = 0 2024-04-16T15:39:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: auth debug = true 2024-04-16T15:39:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on old message = true 2024-04-16T15:39:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on bug = true 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: mon pg warn max object skew = 0 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default size = 2 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow primary affinity = true 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow pg remap = true 2024-04-16T15:39:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on legacy crush tunables = false 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on crush straw calc version zero = false 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on no sortbitwise = false 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on osd down out interval zero = false 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on too few osds = false 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: mon_allow_pool_size_one = true 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: osd default data pool replay window = 5 2024-04-16T15:39:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: mon allow pool delete = true 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: mon cluster log file level = debug 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: debug asserts on shutdown = true 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: mon health detail to clog = false 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: mon host = "172.21.15.71,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]" 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on skipped message = False 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = c5697e20-5e32-4f13-b48e-2f7b8429c0b4 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout:[osd] 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: osd journal size = 100 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub load threshold = 5.0 2024-04-16T15:39:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub max interval = 600 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd mclock profile = high_recovery_ops 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd recover clone overlap = true 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd recovery max chunk = 1048576 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug shutdown = true 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug op order = true 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug verify stray on activate = true 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug trim objects = true 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd open classes on start = true 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug pg log writeout = true 2024-04-16T15:39:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: osd deep scrub update digest min age = 30 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: osd map max advance = 10 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: journal zero on create = true 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: filestore ondisk finisher threads = 3 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: filestore apply finisher threads = 3 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: bdev debug aio = true 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug misdirected ops = true 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore block size = 96636764160 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore fsck on mount = True 2024-04-16T15:39:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluefs = 20 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluestore = 20 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: debug osd = 20 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: debug rocksdb = 10 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd backfillfull_ratio = 0.85 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd full ratio = 0.9 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd nearfull ratio = 0.8 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: osd failsafe full ratio = 0.95 2024-04-16T15:39:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: osd objectstore = bluestore 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: osd op complaint time = 180 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr] 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: debug mgr = 20 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: mgr/telemetry/nag = false 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: debug client = 20 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout:[mon] 2024-04-16T15:39:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: debug paxos = 20 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: mon data avail warn = 5 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: mon mgr mkfs grace = 240 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd reporter subtree level = osd 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd prime pg temp = true 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: # 1m isn't quite enough 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: mon_down_mkfs_grace = 2m 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_filestore_osds = false 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: mon down mkfs grace = 300 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: mon op complaint time = 120 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout:[client] 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: rgw cache enabled = true 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable ops log = true 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable usage log = true 2024-04-16T15:39:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: client mount timeout = 600 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: debug client = 20 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: rados mon op timeout = 900 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: rados osd op timeout = 900 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.a] 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.b] 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.c] 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.a] 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.c] 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.b] 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.d] 2024-04-16T15:39:25.362 INFO:teuthology.orchestra.run.smithi071.stdout:[mds] 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: debug mds = 20 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: debug mds balancer = 20 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: mds debug frag = True 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: mds debug scatterstat = True 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: mds debug zombie log segments = True 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: mds op complaint time = 180 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: mds verify scatter = True 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: osd op complaint time = 180 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: rados mon op timeout = 900 2024-04-16T15:39:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: rados osd op timeout = 900 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-04-16T15:39:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-04-16T15:39:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-04-16T15:39:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-04-16T15:39:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-04-16T15:39:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-04-16T15:39:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-04-16T15:39:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.71,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]" 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on skipped message = False 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = c5697e20-5e32-4f13-b48e-2f7b8429c0b4 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-04-16T15:39:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-04-16T15:39:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-04-16T15:39:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-04-16T15:39:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-04-16T15:39:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-04-16T15:39:25.370 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 20 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 20 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-04-16T15:39:25.371 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 10 2024-04-16T15:39:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-04-16T15:39:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-04-16T15:39:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-04-16T15:39:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-04-16T15:39:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-04-16T15:39:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: osd op complaint time = 180 2024-04-16T15:39:25.372 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.372 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: debug client = 20 2024-04-16T15:39:25.373 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-04-16T15:39:25.374 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-04-16T15:39:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-04-16T15:39:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-16T15:39:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-04-16T15:39:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-04-16T15:39:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-04-16T15:39:25.375 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: mon down mkfs grace = 300 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: mon op complaint time = 120 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-04-16T15:39:25.376 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: client mount timeout = 600 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: debug client = 20 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: rados mon op timeout = 900 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout: rados osd op timeout = 900 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c] 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-04-16T15:39:25.377 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d] 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout:[mds] 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: debug mds = 20 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: debug mds balancer = 20 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: mds debug frag = True 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: mds debug scatterstat = True 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: mds debug zombie log segments = True 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: mds op complaint time = 180 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: mds verify scatter = True 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: osd op complaint time = 180 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: rados mon op timeout = 900 2024-04-16T15:39:25.378 INFO:teuthology.orchestra.run.smithi122.stdout: rados osd op timeout = 900 2024-04-16T15:39:25.380 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-16T15:39:25.380 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-16T15:39:25.479 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-16T15:39:25.479 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:25.479 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-16T15:39:25.494 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:25.494 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-16T15:39:25.550 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:39:25.550 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:25.550 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-16T15:39:25.550 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-16T15:39:25.631 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:25.631 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:25.686 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:39:25.686 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:25.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-16T15:39:25.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-16T15:39:25.729 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:25.729 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:25.784 INFO:tasks.ceph:Setting up mon nodes... 2024-04-16T15:39:25.784 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-16T15:39:25.785 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-16T15:39:25.842 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-16T15:39:25.845 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-16T15:39:25.923 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-16T15:39:25.926 INFO:tasks.ceph:Setting up mds nodes... 2024-04-16T15:39:25.926 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-16T15:39:25.984 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-16T15:39:25.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-16T15:39:26.015 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-16T15:39:26.113 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-16T15:39:26.116 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-16T15:39:26.144 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-16T15:39:26.201 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-16T15:39:26.204 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-16T15:39:26.232 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-16T15:39:26.329 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-16T15:39:26.332 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-16T15:39:26.359 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-16T15:39:26.360 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-16T15:39:26.403 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-16T15:39:26.420 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-16T15:39:26.463 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-16T15:39:26.479 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-16T15:39:26.479 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-16T15:39:26.480 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-16T15:39:26.508 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:26.508 INFO:tasks.ceph:role: osd.0 2024-04-16T15:39:26.508 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:39:26.509 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-16T15:39:26.583 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T15:39:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T15:39:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T15:39:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T15:39:26.584 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T15:39:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-04-16T15:39:26.584 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T15:39:26.584 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T15:39:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T15:39:26.584 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T15:39:27.090 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-04-16T15:39:27.092 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-04-16T15:39:27.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-16T15:39:27.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-16T15:39:27.272 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-16T15:39:27.340 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:27.341 INFO:tasks.ceph:role: osd.1 2024-04-16T15:39:27.341 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:39:27.341 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-16T15:39:27.416 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T15:39:27.416 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T15:39:27.417 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T15:39:27.417 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T15:39:27.417 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T15:39:27.417 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-04-16T15:39:27.417 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T15:39:27.417 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T15:39:27.417 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T15:39:27.417 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T15:39:27.869 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-04-16T15:39:27.871 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-04-16T15:39:27.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-16T15:39:27.956 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-16T15:39:27.988 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-16T15:39:28.056 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:28.056 INFO:tasks.ceph:role: osd.2 2024-04-16T15:39:28.056 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:39:28.056 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-16T15:39:28.132 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T15:39:28.132 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T15:39:28.132 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T15:39:28.132 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T15:39:28.132 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T15:39:28.132 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-04-16T15:39:28.132 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T15:39:28.132 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T15:39:28.133 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T15:39:28.133 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T15:39:28.619 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-04-16T15:39:28.621 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-04-16T15:39:28.621 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-16T15:39:28.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-16T15:39:28.785 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-16T15:39:28.854 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:28.855 INFO:tasks.ceph:role: osd.3 2024-04-16T15:39:28.855 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:39:28.855 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-16T15:39:28.931 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T15:39:28.931 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T15:39:28.931 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T15:39:28.931 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T15:39:28.931 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T15:39:28.931 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2024-04-16T15:39:28.931 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T15:39:28.932 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T15:39:28.932 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T15:39:28.932 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T15:39:29.446 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2024-04-16T15:39:29.448 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2024-04-16T15:39:29.448 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-16T15:39:29.576 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-16T15:39:29.612 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:29.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:29.713+0000 7fe2613f6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-16T15:39:29.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:29.713+0000 7fe2613f6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-16T15:39:29.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:29.713+0000 7fe2613f6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-16T15:39:29.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:29.713+0000 7fe2613f6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-16T15:39:29.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:29.713+0000 7fe2613f6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-16T15:39:29.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:29.714+0000 7fe2613f6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-16T15:39:32.566 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-16T15:39:32.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:32.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:32.695+0000 7f1ea2448640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-16T15:39:32.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:32.696+0000 7f1ea2448640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-16T15:39:32.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:32.696+0000 7f1ea2448640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-16T15:39:32.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:32.696+0000 7f1ea2448640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-16T15:39:32.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:32.696+0000 7f1ea2448640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-16T15:39:32.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:32.696+0000 7f1ea2448640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-16T15:39:35.544 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-16T15:39:35.577 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:35.670+0000 7f4049b19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-16T15:39:35.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:35.671+0000 7f4049b19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-16T15:39:35.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:35.671+0000 7f4049b19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-16T15:39:35.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:35.671+0000 7f4049b19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-16T15:39:35.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:35.671+0000 7f4049b19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-16T15:39:35.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:35.671+0000 7f4049b19640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-16T15:39:38.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-16T15:39:38.557 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:38.650+0000 7f218d611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-16T15:39:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:38.650+0000 7f218d611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-16T15:39:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:38.650+0000 7f218d611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-16T15:39:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:38.650+0000 7f218d611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-16T15:39:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:38.650+0000 7f218d611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-16T15:39:38.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:38.650+0000 7f218d611640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-16T15:39:41.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-16T15:39:41.533 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-16T15:39:41.567 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:41.568 INFO:tasks.ceph:role: osd.4 2024-04-16T15:39:41.568 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:39:41.568 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-16T15:39:41.648 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T15:39:41.648 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T15:39:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T15:39:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T15:39:41.649 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T15:39:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-04-16T15:39:41.649 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T15:39:41.649 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T15:39:41.649 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T15:39:41.649 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T15:39:42.174 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-04-16T15:39:42.177 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-04-16T15:39:42.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-16T15:39:42.312 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-16T15:39:42.348 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-16T15:39:42.416 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:42.417 INFO:tasks.ceph:role: osd.5 2024-04-16T15:39:42.417 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:39:42.417 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-16T15:39:42.492 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T15:39:42.492 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T15:39:42.492 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T15:39:42.492 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T15:39:42.492 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T15:39:42.493 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-04-16T15:39:42.493 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T15:39:42.493 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T15:39:42.493 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T15:39:42.493 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T15:39:42.932 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-04-16T15:39:42.935 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-04-16T15:39:42.935 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-16T15:39:43.028 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-16T15:39:43.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-16T15:39:43.127 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:43.127 INFO:tasks.ceph:role: osd.6 2024-04-16T15:39:43.127 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:39:43.128 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T15:39:43.203 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T15:39:43.684 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-04-16T15:39:43.686 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-04-16T15:39:43.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-16T15:39:43.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-16T15:39:43.875 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-16T15:39:43.943 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-16T15:39:43.943 INFO:tasks.ceph:role: osd.7 2024-04-16T15:39:43.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:39:43.943 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-16T15:39:44.019 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-16T15:39:44.019 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-04-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-16T15:39:44.521 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-04-16T15:39:44.523 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-04-16T15:39:44.524 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-16T15:39:44.647 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-16T15:39:44.681 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:44.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:44.801+0000 7fe37d51b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-16T15:39:44.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:44.802+0000 7fe37d51b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-16T15:39:44.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:44.802+0000 7fe37d51b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-16T15:39:44.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:44.802+0000 7fe37d51b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-16T15:39:44.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:44.802+0000 7fe37d51b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-16T15:39:44.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:44.802+0000 7fe37d51b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-16T15:39:47.648 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-16T15:39:47.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:47.781+0000 7f50b4c68640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-16T15:39:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:47.781+0000 7f50b4c68640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-16T15:39:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:47.781+0000 7f50b4c68640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-16T15:39:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:47.781+0000 7f50b4c68640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-16T15:39:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:47.781+0000 7f50b4c68640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-16T15:39:47.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:47.782+0000 7f50b4c68640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-16T15:39:50.631 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-16T15:39:50.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:50.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:50.759+0000 7f699bd06640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-16T15:39:50.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:50.759+0000 7f699bd06640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-16T15:39:50.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:50.759+0000 7f699bd06640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-16T15:39:50.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:50.759+0000 7f699bd06640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-16T15:39:50.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:50.759+0000 7f699bd06640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-16T15:39:50.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:50.760+0000 7f699bd06640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-16T15:39:53.302 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:39:53.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:39:53.609 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-16T15:39:53.639 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:53.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:53.728+0000 7facf9cd5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-16T15:39:53.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:53.728+0000 7facf9cd5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-16T15:39:53.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:53.728+0000 7facf9cd5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-16T15:39:53.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:53.728+0000 7facf9cd5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-16T15:39:53.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:53.728+0000 7facf9cd5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-16T15:39:53.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:39:53.729+0000 7facf9cd5640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-16T15:39:56.572 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-16T15:39:56.605 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-16T15:39:56.605 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:56.606 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-16T15:39:56.639 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:56.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-16T15:39:56.711 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:56.711 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-16T15:39:56.782 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:56.782 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-16T15:39:56.852 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:56.853 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-16T15:39:56.922 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:56.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-16T15:39:56.990 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:56.990 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-16T15:39:57.059 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:57.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-16T15:39:57.093 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:57.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-16T15:39:57.167 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:57.167 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-16T15:39:57.240 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:57.240 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-16T15:39:57.310 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:57.310 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-16T15:39:57.380 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:57.380 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-16T15:39:57.450 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:57.450 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-16T15:39:57.519 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:57.519 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-16T15:39:57.536 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:57.536 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-16T15:39:57.576 INFO:tasks.ceph:Adding keys to all mons... 2024-04-16T15:39:57.576 DEBUG:teuthology.orchestra.run.smithi071:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-16T15:39:57.578 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-16T15:39:57.620 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.y] 2024-04-16T15:39:57.620 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCtmx5mmB1aMhAAFQuNW0KOn1oJcsNCxMKeXQ== 2024-04-16T15:39:57.620 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.a] 2024-04-16T15:39:57.620 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCtmx5maSbSOhAAEIlR48PaHF+TNphBw608dQ== 2024-04-16T15:39:57.620 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.c] 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCumx5mPxjpBhAAgnn0yvcH+OHJGkmM+wREeQ== 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.0] 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCxmx5mEtyQKhAAvTQwwrkkdepLFbdaJ3z6+w== 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.1] 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQC0mx5mizaHKRAA9ods0Zji7MgKtrqv41vTtg== 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.2] 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQC3mx5mXzwKKBAANUOg3ynvzO62gVGupvq0kw== 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.3] 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQC6mx5mqNvLJhAA3Qj3Joh87KtLHV9xbJ7vWA== 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.x] 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCtmx5mwREuNxAA0isz1n+5hfl4bBvzO3UOuA== 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.b] 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCumx5mYc8lDBAAkxIO7XDmrv0CRfaWHUGQ6A== 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.d] 2024-04-16T15:39:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCumx5meQ/JExAAvSY3b7bj83gzM9ghd9Qkrg== 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.4] 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDAmx5mb77YLxAAB1NQ96N0oEq8guwEeOcSbQ== 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.5] 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDDmx5mabyhLhAA0vq9zxx4L1s0w6vLSADfRw== 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.6] 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDGmx5mVKFQLRAAUdDofDpmWZykpPbufPrVVw== 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.7] 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDJmx5m7V13KxAAb+B6sdFAwn/o48wloWR0sg== 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCumx5me0E3GBAAUrYP3hRI128xd+yf1A5VaA== 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout:[client.1] 2024-04-16T15:39:57.622 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCumx5m++vGGxAAkTQKAgscEljR9aCDLR3Oww== 2024-04-16T15:39:57.644 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.y] 2024-04-16T15:39:57.644 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCtmx5mmB1aMhAAFQuNW0KOn1oJcsNCxMKeXQ== 2024-04-16T15:39:57.644 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-04-16T15:39:57.644 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCtmx5maSbSOhAAEIlR48PaHF+TNphBw608dQ== 2024-04-16T15:39:57.644 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c] 2024-04-16T15:39:57.644 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCumx5mPxjpBhAAgnn0yvcH+OHJGkmM+wREeQ== 2024-04-16T15:39:57.644 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-04-16T15:39:57.644 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCxmx5mEtyQKhAAvTQwwrkkdepLFbdaJ3z6+w== 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQC0mx5mizaHKRAA9ods0Zji7MgKtrqv41vTtg== 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQC3mx5mXzwKKBAANUOg3ynvzO62gVGupvq0kw== 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQC6mx5mqNvLJhAA3Qj3Joh87KtLHV9xbJ7vWA== 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCtmx5mwREuNxAA0isz1n+5hfl4bBvzO3UOuA== 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCumx5mYc8lDBAAkxIO7XDmrv0CRfaWHUGQ6A== 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d] 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCumx5meQ/JExAAvSY3b7bj83gzM9ghd9Qkrg== 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-04-16T15:39:57.645 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQDAmx5mb77YLxAAB1NQ96N0oEq8guwEeOcSbQ== 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQDDmx5mabyhLhAA0vq9zxx4L1s0w6vLSADfRw== 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.6] 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQDGmx5mVKFQLRAAUdDofDpmWZykpPbufPrVVw== 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.7] 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQDJmx5m7V13KxAAb+B6sdFAwn/o48wloWR0sg== 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCumx5me0E3GBAAUrYP3hRI128xd+yf1A5VaA== 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout:[client.1] 2024-04-16T15:39:57.646 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCumx5m++vGGxAAkTQKAgscEljR9aCDLR3Oww== 2024-04-16T15:39:57.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-16T15:39:57.666 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-16T15:39:57.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-16T15:39:57.744 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-16T15:39:57.800 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-16T15:39:57.802 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-16T15:39:57.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:57.862 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:57.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:57.918 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:57.971 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:57.974 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.027 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.083 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-16T15:39:58.085 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-16T15:39:58.139 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-16T15:39:58.141 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-16T15:39:58.197 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-16T15:39:58.199 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-16T15:39:58.256 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.258 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.314 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.368 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.370 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-16T15:39:58.479 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T15:39:58.481 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T15:39:58.538 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T15:39:58.540 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-16T15:39:58.598 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-16T15:39:58.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-16T15:39:58.626 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-16T15:39:58.725 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-16T15:39:58.754 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-16T15:39:58.783 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-16T15:39:58.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-16T15:39:58.913 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-16T15:39:58.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-16T15:39:59.080 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-16T15:39:59.109 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:59.112 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-16T15:39:59.164 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-16T15:39:59.164 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-16T15:39:59.164 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-16T15:39:59.168 INFO:tasks.ceph.mon.a:Started 2024-04-16T15:39:59.168 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-16T15:39:59.168 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-16T15:39:59.206 INFO:tasks.ceph.mon.b:Started 2024-04-16T15:39:59.206 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-16T15:39:59.206 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-16T15:39:59.208 INFO:tasks.ceph.mon.c:Started 2024-04-16T15:39:59.208 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-16T15:39:59.209 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-16T15:39:59.209 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-16T15:39:59.211 INFO:tasks.ceph.mgr.y:Started 2024-04-16T15:39:59.211 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-16T15:39:59.211 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-16T15:39:59.213 INFO:tasks.ceph.mgr.x:Started 2024-04-16T15:39:59.214 INFO:tasks.ceph:Setting crush tunables to default 2024-04-16T15:39:59.214 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph osd crush tunables default 2024-04-16T15:39:59.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.319+0000 7f2185247640 1 Processor -- start 2024-04-16T15:39:59.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.319+0000 7f2185247640 1 -- start start 2024-04-16T15:39:59.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f2185247640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 0x7f2180159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f2185247640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 0x7f2180153430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f2185247640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2180155c70 con 0x7f21801566f0 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f2185247640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f21800c7220 con 0x7f2180152fa0 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f2185247640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f21800c7830 con 0x7f21800ca690 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217f59e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 0x7f2180153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217f59e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 0x7f2180153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43070/0 (socket says 172.21.15.71:43070) 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217f59e640 1 -- 172.21.15.71:0/506437238 learned_addr learned my addr 172.21.15.71:0/506437238 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217f59e640 1 -- 172.21.15.71:0/506437238 >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 msgr2=0x7f2180153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217f59e640 1 -- 172.21.15.71:0/506437238 >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 msgr2=0x7f2180153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217f59e640 1 --2- 172.21.15.71:0/506437238 >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 0x7f2180153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217f59e640 1 --2- 172.21.15.71:0/506437238 >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 0x7f2180153430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217ed9d640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 0x7f2180159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:39:59.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217ed9d640 1 -- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 msgr2=0x7f2180159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-16T15:39:59.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217ed9d640 1 -- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 msgr2=0x7f2180159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-16T15:39:59.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217ed9d640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 0x7f2180159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T15:39:59.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.320+0000 7f217ed9d640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 0x7f2180159bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T15:39:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 932174708 0 0) 0x7f2174001110 con 0x7f21801566f0 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2168003520 con 0x7f21801566f0 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2117081635 0 0) 0x7f2168003520 con 0x7f21801566f0 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 msgr2=0x7f2180159bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 0x7f2180159bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 msgr2=0x7f2180153430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 --2- 172.21.15.71:0/506437238 >> v2:172.21.15.71:3300/0 conn(0x7f2180152fa0 0x7f2180153430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21800c7b10 con 0x7f21801566f0 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f2174001840 con 0x7f21801566f0 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 >> v1:172.21.15.122:6789/0 conn(0x7f21801566f0 legacy=0x7f21800ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21680039b0 0x7f2168003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 0x7f2168004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168004d40 0x7f21680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:39:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21800c7830 con 0x7f21680039b0 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f216800ba70 con 0x7f2168004d40 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.485+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f216800bc70 con 0x7f21800ca690 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.486+0000 7f217ed9d640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21680039b0 0x7f2168003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.486+0000 7f217f59e640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168004d40 0x7f21680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.486+0000 7f217e59c640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 0x7f2168004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.486+0000 7f217ed9d640 1 -- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 msgr2=0x7f2168004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.486+0000 7f217ed9d640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21800ca690 0x7f2168004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.486+0000 7f217ed9d640 1 -- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168004d40 msgr2=0x7f21680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.486+0000 7f217ed9d640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168004d40 0x7f21680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.486+0000 7f217ed9d640 1 -- 172.21.15.71:0/506437238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2168003520 con 0x7f21680039b0 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.487+0000 7f217ed9d640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21680039b0 0x7f2168003dd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f217000b9e0 tx=0x7f2168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4f3357b42e6566 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.487+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170036020 con 0x7f21680039b0 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.487+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2170032620 con 0x7f21680039b0 2024-04-16T15:39:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.487+0000 7f217dd9b640 1 -- 172.21.15.71:0/506437238 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217000b5e0 con 0x7f21680039b0 2024-04-16T15:39:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.487+0000 7f2185247640 1 -- 172.21.15.71:0/506437238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21680039b0 msgr2=0x7f2168003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:39:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.488+0000 7f2185247640 1 --2- 172.21.15.71:0/506437238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21680039b0 0x7f2168003dd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f217000b9e0 tx=0x7f2168005830 comp rx=0 tx=0).stop 2024-04-16T15:39:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.488+0000 7f217f59e640 1 -- 172.21.15.71:0/506437238 reap_dead start 2024-04-16T15:39:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.488+0000 7f2185247640 1 -- 172.21.15.71:0/506437238 shutdown_connections 2024-04-16T15:39:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.488+0000 7f2185247640 1 -- 172.21.15.71:0/506437238 >> 172.21.15.71:0/506437238 conn(0x7f21800ad1e0 msgr2=0x7f21800ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:39:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.488+0000 7f2185247640 1 -- 172.21.15.71:0/506437238 shutdown_connections 2024-04-16T15:39:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.488+0000 7f2185247640 1 -- 172.21.15.71:0/506437238 wait complete. 2024-04-16T15:39:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.489+0000 7f2185247640 1 Processor -- start 2024-04-16T15:39:59.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.489+0000 7f2185247640 1 -- start start 2024-04-16T15:39:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.490+0000 7f2185247640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21680039b0 0x7f21800ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:39:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.490+0000 7f2185247640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21801ef5f0 0x7f2180150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:39:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f2185247640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f218014fed0 0x7f2180148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:39:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f2185247640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2180156b70 con 0x7f218014fed0 2024-04-16T15:39:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f2185247640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21801561d0 con 0x7f21801ef5f0 2024-04-16T15:39:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f2185247640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f21800bed50 con 0x7f21680039b0 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217f59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f218014fed0 0x7f2180148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21801ef5f0 0x7f2180150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217ed9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21680039b0 0x7f21800ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21801ef5f0 0x7f2180150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:42916/0 (socket says 172.21.15.71:42916) 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217f59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f218014fed0 0x7f2180148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43086/0 (socket says 172.21.15.71:43086) 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217e59c640 1 -- 172.21.15.71:0/1840223394 learned_addr learned my addr 172.21.15.71:0/1840223394 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217f59e640 1 -- 172.21.15.71:0/1840223394 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21680039b0 msgr2=0x7f21800ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217f59e640 1 --2- 172.21.15.71:0/1840223394 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21680039b0 0x7f21800ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217f59e640 1 -- 172.21.15.71:0/1840223394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21801ef5f0 msgr2=0x7f2180150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217f59e640 1 --2- 172.21.15.71:0/1840223394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21801ef5f0 0x7f2180150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217f59e640 1 -- 172.21.15.71:0/1840223394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2180159140 con 0x7f218014fed0 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.491+0000 7f217f59e640 1 --2- 172.21.15.71:0/1840223394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f218014fed0 0x7f2180148ab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f21800744e0 tx=0x7f2178007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.492+0000 7f21677fe640 1 -- 172.21.15.71:0/1840223394 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217801b070 con 0x7f218014fed0 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.492+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21801580f0 con 0x7f218014fed0 2024-04-16T15:39:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.492+0000 7f21677fe640 1 -- 172.21.15.71:0/1840223394 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217800a500 con 0x7f218014fed0 2024-04-16T15:39:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.492+0000 7f21677fe640 1 -- 172.21.15.71:0/1840223394 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217800a6b0 con 0x7f218014fed0 2024-04-16T15:39:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.492+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2180158dd0 con 0x7f218014fed0 2024-04-16T15:39:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.492+0000 7f21677fe640 1 -- 172.21.15.71:0/1840223394 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f217800f800 con 0x7f218014fed0 2024-04-16T15:39:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.492+0000 7f21677fe640 1 -- 172.21.15.71:0/1840223394 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2178004370 con 0x7f218014fed0 2024-04-16T15:39:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.493+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2180068be0 con 0x7f218014fed0 2024-04-16T15:39:59.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.495+0000 7f21677fe640 1 -- 172.21.15.71:0/1840223394 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f217800fc80 con 0x7f218014fed0 2024-04-16T15:39:59.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.562+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2180155a50 con 0x7f218014fed0 2024-04-16T15:39:59.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.563+0000 7f21677fe640 1 -- 172.21.15.71:0/1840223394 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f217800fe60 con 0x7f218014fed0 2024-04-16T15:39:59.562 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-04-16T15:39:59.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f218014fed0 msgr2=0x7f2180148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:39:59.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 --2- 172.21.15.71:0/1840223394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f218014fed0 0x7f2180148ab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f21800744e0 tx=0x7f2178007f40 comp rx=0 tx=0).stop 2024-04-16T15:39:59.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 shutdown_connections 2024-04-16T15:39:59.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 --2- 172.21.15.71:0/1840223394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f218014fed0 0x7f2180148ab0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 --2- 172.21.15.71:0/1840223394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21801ef5f0 0x7f2180150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 --2- 172.21.15.71:0/1840223394 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21680039b0 0x7f21800ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:39:59.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 >> 172.21.15.71:0/1840223394 conn(0x7f21800ad1e0 msgr2=0x7f218014cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:39:59.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 shutdown_connections 2024-04-16T15:39:59.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:39:59.564+0000 7f2185247640 1 -- 172.21.15.71:0/1840223394 wait complete. 2024-04-16T15:39:59.574 INFO:tasks.ceph:check_enable_crimson: False 2024-04-16T15:39:59.574 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-16T15:39:59.575 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:59.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-16T15:39:59.603 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:59.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-16T15:39:59.672 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:59.672 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-16T15:39:59.740 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:39:59.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-16T15:39:59.809 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:59.809 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-16T15:39:59.837 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:59.838 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-16T15:39:59.906 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:59.906 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-16T15:39:59.964 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:39:59.965+0000 7fe1d6e37040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T15:39:59.975 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:39:59.975 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-16T15:39:59.978 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:39:59.980+0000 7f4d70e42040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T15:40:00.020 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:00.021+0000 7fe1d6e37040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T15:40:00.032 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:00.034+0000 7f4d70e42040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T15:40:00.045 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 76b3f430-8289-42d6-9cb4-93fba8f0da14 0 2024-04-16T15:40:00.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.188+0000 7f54c3647640 1 Processor -- start 2024-04-16T15:40:00.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.188+0000 7f54c3647640 1 -- start start 2024-04-16T15:40:00.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.188+0000 7f54c3647640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f54bc0c52e0 0x7f54bc0c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.188+0000 7f54c3647640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f54bc0c74f0 0x7f54bc06c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.188+0000 7f54c3647640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54bc072770 con 0x7f54bc0c3b10 2024-04-16T15:40:00.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.188+0000 7f54c3647640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f54bc062cd0 con 0x7f54bc0c74f0 2024-04-16T15:40:00.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c3647640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f54bc0a7510 con 0x7f54bc0c52e0 2024-04-16T15:40:00.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c0be2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f54bc0c52e0 0x7f54bc0c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c13e3640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f54bc0c3b10 0x7f54bc155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:41888/0 (socket says 172.21.15.122:41888) 2024-04-16T15:40:00.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c0be2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f54bc0c52e0 0x7f54bc0c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32768/0 (socket says 172.21.15.122:32768) 2024-04-16T15:40:00.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c13e3640 1 -- 172.21.15.122:0/3083345846 learned_addr learned my addr 172.21.15.122:0/3083345846 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-04-16T15:40:00.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c1be4640 1 --2- 172.21.15.122:0/3083345846 >> v2:172.21.15.71:3300/0 conn(0x7f54bc0c74f0 0x7f54bc06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c0be2640 1 -- 172.21.15.122:0/3083345846 >> v2:172.21.15.71:3300/0 conn(0x7f54bc0c74f0 msgr2=0x7f54bc06c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:00.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c0be2640 1 --2- 172.21.15.122:0/3083345846 >> v2:172.21.15.71:3300/0 conn(0x7f54bc0c74f0 0x7f54bc06c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c0be2640 1 -- 172.21.15.122:0/3083345846 >> v1:172.21.15.122:6789/0 conn(0x7f54bc0c3b10 legacy=0x7f54bc155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.189+0000 7f54c0be2640 1 -- 172.21.15.122:0/3083345846 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc062f20 con 0x7f54bc0c52e0 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.190+0000 7f54c0be2640 1 --2- 172.21.15.122:0/3083345846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c52e0 0x7f54bc0c1b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0a7070 tx=0x7f54b002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=677cdf27504b28ab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.190+0000 7f54affff640 1 -- 172.21.15.122:0/3083345846 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0035020 con 0x7f54bc0c52e0 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.190+0000 7f54affff640 1 -- 172.21.15.122:0/3083345846 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b0031830 con 0x7f54bc0c52e0 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.190+0000 7f54affff640 1 -- 172.21.15.122:0/3083345846 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0031c00 con 0x7f54bc0c52e0 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.191+0000 7f54c3647640 1 -- 172.21.15.122:0/3083345846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c52e0 msgr2=0x7f54bc0c1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.191+0000 7f54c3647640 1 --2- 172.21.15.122:0/3083345846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c52e0 0x7f54bc0c1b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0a7070 tx=0x7f54b002b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.191+0000 7f54c3647640 1 -- 172.21.15.122:0/3083345846 shutdown_connections 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.191+0000 7f54c3647640 1 --2- 172.21.15.122:0/3083345846 >> v2:172.21.15.71:3300/0 conn(0x7f54bc0c74f0 0x7f54bc06c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.191+0000 7f54c3647640 1 --2- 172.21.15.122:0/3083345846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c52e0 0x7f54bc0c1b30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.191+0000 7f54c3647640 1 -- 172.21.15.122:0/3083345846 >> 172.21.15.122:0/3083345846 conn(0x7f54bc0c4640 msgr2=0x7f54bc0c4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:00.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.191+0000 7f54c3647640 1 -- 172.21.15.122:0/3083345846 shutdown_connections 2024-04-16T15:40:00.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.191+0000 7f54c3647640 1 -- 172.21.15.122:0/3083345846 wait complete. 2024-04-16T15:40:00.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.192+0000 7f54c3647640 1 Processor -- start 2024-04-16T15:40:00.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.192+0000 7f54c3647640 1 -- start start 2024-04-16T15:40:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.192+0000 7f54c3647640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c3b10 0x7f54bc06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.192+0000 7f54c3647640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54bc0c52e0 0x7f54bc1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.192+0000 7f54c3647640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc0c74f0 0x7f54bc150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c3647640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc156350 con 0x7f54bc0c74f0 2024-04-16T15:40:00.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c3647640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc0c5d20 con 0x7f54bc0c3b10 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c3647640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f54bc0c3ef0 con 0x7f54bc0c52e0 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c3b10 0x7f54bc06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c3b10 0x7f54bc06bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32780/0 (socket says 172.21.15.122:32780) 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 -- 172.21.15.122:0/2922091152 learned_addr learned my addr 172.21.15.122:0/2922091152 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c0be2640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54bc0c52e0 0x7f54bc1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c1be4640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc0c74f0 0x7f54bc150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 -- 172.21.15.122:0/2922091152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54bc0c52e0 msgr2=0x7f54bc1556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54bc0c52e0 0x7f54bc1556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 -- 172.21.15.122:0/2922091152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc0c74f0 msgr2=0x7f54bc150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc0c74f0 0x7f54bc150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 -- 172.21.15.122:0/2922091152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc154ba0 con 0x7f54bc0c3b10 2024-04-16T15:40:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c13e3640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c3b10 0x7f54bc06bfe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c400 tx=0x7f54b800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54adffb640 1 -- 172.21.15.122:0/2922091152 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8017070 con 0x7f54bc0c3b10 2024-04-16T15:40:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54adffb640 1 -- 172.21.15.122:0/2922091152 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b8005ce0 con 0x7f54bc0c3b10 2024-04-16T15:40:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54bc154900 con 0x7f54bc0c3b10 2024-04-16T15:40:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54adffb640 1 -- 172.21.15.122:0/2922091152 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80072f0 con 0x7f54bc0c3b10 2024-04-16T15:40:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.193+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54bc06c520 con 0x7f54bc0c3b10 2024-04-16T15:40:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.194+0000 7f54adffb640 1 -- 172.21.15.122:0/2922091152 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54b801a070 con 0x7f54bc0c3b10 2024-04-16T15:40:00.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.194+0000 7f54adffb640 1 -- 172.21.15.122:0/2922091152 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f54b8012400 con 0x7f54bc0c3b10 2024-04-16T15:40:00.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.194+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54bc0c1880 con 0x7f54bc0c3b10 2024-04-16T15:40:00.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.196+0000 7f54adffb640 1 -- 172.21.15.122:0/2922091152 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f54bc0c1880 con 0x7f54bc0c3b10 2024-04-16T15:40:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.268+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "76b3f430-8289-42d6-9cb4-93fba8f0da14", "id": 0} v 0) -- 0x7f54bc0ab290 con 0x7f54bc0c3b10 2024-04-16T15:40:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.271+0000 7f54adffb640 1 -- 172.21.15.122:0/2922091152 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "76b3f430-8289-42d6-9cb4-93fba8f0da14", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f54b8021020 con 0x7f54bc0c3b10 2024-04-16T15:40:00.271 INFO:teuthology.orchestra.run.smithi122.stdout:0 2024-04-16T15:40:00.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c3b10 msgr2=0x7f54bc06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c3b10 0x7f54bc06bfe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c400 tx=0x7f54b800c8c0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 shutdown_connections 2024-04-16T15:40:00.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc0c74f0 0x7f54bc150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54bc0c52e0 0x7f54bc1556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 --2- 172.21.15.122:0/2922091152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54bc0c3b10 0x7f54bc06bfe0 secure :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f54b800c400 tx=0x7f54b800c8c0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 >> 172.21.15.122:0/2922091152 conn(0x7f54bc0c4640 msgr2=0x7f54bc072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 shutdown_connections 2024-04-16T15:40:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.272+0000 7f54c3647640 1 -- 172.21.15.122:0/2922091152 wait complete. 2024-04-16T15:40:00.283 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9 1 2024-04-16T15:40:00.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.385+0000 7fd67fa35640 1 Processor -- start 2024-04-16T15:40:00.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67fa35640 1 -- start start 2024-04-16T15:40:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67fa35640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fd6781514b0 0x7fd6780c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67fa35640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd678157300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67fa35640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6780c55b0 con 0x7fd6780c4b10 2024-04-16T15:40:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67fa35640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fd6780be6e0 con 0x7fd6781514b0 2024-04-16T15:40:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67fa35640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd6780c5890 con 0x7fd6780c8ee0 2024-04-16T15:40:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67cfd0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd678157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67dfd2640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd6780c4b10 0x7fd678157840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:41898/0 (socket says 172.21.15.122:41898) 2024-04-16T15:40:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.386+0000 7fd67dfd2640 1 -- 172.21.15.122:0/2925587911 learned_addr learned my addr 172.21.15.122:0/2925587911 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-04-16T15:40:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.387+0000 7fd67d7d1640 1 --2- 172.21.15.122:0/2925587911 >> v2:172.21.15.71:3300/0 conn(0x7fd6781514b0 0x7fd6780c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.387+0000 7fd66ffff640 1 -- 172.21.15.122:0/2925587911 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408608231 0 0) 0x7fd6780c55b0 con 0x7fd6780c4b10 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.387+0000 7fd66ffff640 1 -- 172.21.15.122:0/2925587911 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd664003520 con 0x7fd6780c4b10 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.387+0000 7fd67cfd0640 1 -- 172.21.15.122:0/2925587911 >> v2:172.21.15.71:3300/0 conn(0x7fd6781514b0 msgr2=0x7fd6780c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.387+0000 7fd67cfd0640 1 --2- 172.21.15.122:0/2925587911 >> v2:172.21.15.71:3300/0 conn(0x7fd6781514b0 0x7fd6780c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.387+0000 7fd67cfd0640 1 -- 172.21.15.122:0/2925587911 >> v1:172.21.15.122:6789/0 conn(0x7fd6780c4b10 legacy=0x7fd678157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.387+0000 7fd67cfd0640 1 -- 172.21.15.122:0/2925587911 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6780c5b70 con 0x7fd6780c8ee0 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67cfd0640 1 --2- 172.21.15.122:0/2925587911 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd678157300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6680097a0 tx=0x7fd66802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2df8dfe5a4bafdc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd66ffff640 1 -- 172.21.15.122:0/2925587911 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680315a0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd66ffff640 1 -- 172.21.15.122:0/2925587911 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668037070 con 0x7fd6780c8ee0 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd66ffff640 1 -- 172.21.15.122:0/2925587911 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668031960 con 0x7fd6780c8ee0 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67fa35640 1 -- 172.21.15.122:0/2925587911 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 msgr2=0x7fd678157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67fa35640 1 --2- 172.21.15.122:0/2925587911 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd678157300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6680097a0 tx=0x7fd66802b840 comp rx=0 tx=0).stop 2024-04-16T15:40:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67fa35640 1 -- 172.21.15.122:0/2925587911 shutdown_connections 2024-04-16T15:40:00.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67fa35640 1 --2- 172.21.15.122:0/2925587911 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd678157300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67fa35640 1 --2- 172.21.15.122:0/2925587911 >> v2:172.21.15.71:3300/0 conn(0x7fd6781514b0 0x7fd6780c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67fa35640 1 -- 172.21.15.122:0/2925587911 >> 172.21.15.122:0/2925587911 conn(0x7fd6780ab3b0 msgr2=0x7fd6780ab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:00.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67fa35640 1 -- 172.21.15.122:0/2925587911 shutdown_connections 2024-04-16T15:40:00.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.388+0000 7fd67fa35640 1 -- 172.21.15.122:0/2925587911 wait complete. 2024-04-16T15:40:00.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67fa35640 1 Processor -- start 2024-04-16T15:40:00.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67fa35640 1 -- start start 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67fa35640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6780c4b10 0x7fd6781ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67fa35640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd6781ed460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67fa35640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6781514b0 0x7fd67806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67fa35640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780ae810 con 0x7fd6780c4b10 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67fa35640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780ae690 con 0x7fd6781514b0 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67fa35640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd6780c99e0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67dfd2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6781514b0 0x7fd67806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67cfd0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd6781ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67cfd0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd6781ed460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:37030/0 (socket says 172.21.15.122:37030) 2024-04-16T15:40:00.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67cfd0640 1 -- 172.21.15.122:0/120702794 learned_addr learned my addr 172.21.15.122:0/120702794 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67dfd2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6781514b0 0x7fd67806fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32794/0 (socket says 172.21.15.122:32794) 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.390+0000 7fd67d7d1640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6780c4b10 0x7fd6781ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67cfd0640 1 -- 172.21.15.122:0/120702794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6781514b0 msgr2=0x7fd67806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67cfd0640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6781514b0 0x7fd67806fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67cfd0640 1 -- 172.21.15.122:0/120702794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6780c4b10 msgr2=0x7fd6781ecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67cfd0640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6780c4b10 0x7fd6781ecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67cfd0640 1 -- 172.21.15.122:0/120702794 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6781edbc0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67dfd2640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6781514b0 0x7fd67806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67cfd0640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd6781ed460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd668034040 tx=0x7fd6680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd66dffb640 1 -- 172.21.15.122:0/120702794 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668032630 con 0x7fd6780c8ee0 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd66dffb640 1 -- 172.21.15.122:0/120702794 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6680327f0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67806ed80 con 0x7fd6780c8ee0 2024-04-16T15:40:00.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd66dffb640 1 -- 172.21.15.122:0/120702794 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668032be0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67806f340 con 0x7fd6780c8ee0 2024-04-16T15:40:00.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd66dffb640 1 -- 172.21.15.122:0/120702794 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd66804e7c0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.391+0000 7fd66dffb640 1 -- 172.21.15.122:0/120702794 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd66802f030 con 0x7fd6780c8ee0 2024-04-16T15:40:00.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.392+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6780c83e0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.394+0000 7fd66dffb640 1 -- 172.21.15.122:0/120702794 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd6781edbc0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.467+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9", "id": 1} v 0) -- 0x7fd678068be0 con 0x7fd6780c8ee0 2024-04-16T15:40:00.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.470+0000 7fd66dffb640 1 -- 172.21.15.122:0/120702794 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd668040070 con 0x7fd6780c8ee0 2024-04-16T15:40:00.470 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-04-16T15:40:00.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 msgr2=0x7fd6781ed460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd6781ed460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd668034040 tx=0x7fd6680046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 shutdown_connections 2024-04-16T15:40:00.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6781514b0 0x7fd67806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd6780c8ee0 0x7fd6781ed460 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 --2- 172.21.15.122:0/120702794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6780c4b10 0x7fd6781ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 >> 172.21.15.122:0/120702794 conn(0x7fd6780ab3b0 msgr2=0x7fd6780a99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 shutdown_connections 2024-04-16T15:40:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.471+0000 7fd67fa35640 1 -- 172.21.15.122:0/120702794 wait complete. 2024-04-16T15:40:00.483 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 2eec121c-f3b5-4f74-8b1e-f389c7d99f3b 2 2024-04-16T15:40:00.585 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:00.585+0000 7fe1d6e37040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T15:40:00.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.586+0000 7fb25e00d640 1 Processor -- start 2024-04-16T15:40:00.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.586+0000 7fb25e00d640 1 -- start start 2024-04-16T15:40:00.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb25e00d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb25814cd40 0x7fb258073b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb25e00d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb25806f950 0x7fb25806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb25e00d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb258152c30 con 0x7fb258073570 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb25e00d640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fb2580c5e30 con 0x7fb25806f950 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb25e00d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb258152f10 con 0x7fb25814cd40 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb2577fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb25814cd40 0x7fb258073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb2577fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb25814cd40 0x7fb258073b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32810/0 (socket says 172.21.15.122:32810) 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb2577fe640 1 -- 172.21.15.122:0/184898590 learned_addr learned my addr 172.21.15.122:0/184898590 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb24ffff640 1 --1- 172.21.15.122:0/184898590 >> v1:172.21.15.122:6789/0 conn(0x7fb258073570 0x7fb25806d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:41902/0 (socket says 172.21.15.122:41902) 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.587+0000 7fb257fff640 1 --2- 172.21.15.122:0/184898590 >> v2:172.21.15.71:3300/0 conn(0x7fb25806f950 0x7fb25806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.588+0000 7fb2577fe640 1 -- 172.21.15.122:0/184898590 >> v2:172.21.15.71:3300/0 conn(0x7fb25806f950 msgr2=0x7fb25806fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.588+0000 7fb2577fe640 1 --2- 172.21.15.122:0/184898590 >> v2:172.21.15.71:3300/0 conn(0x7fb25806f950 0x7fb25806fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.588+0000 7fb2577fe640 1 -- 172.21.15.122:0/184898590 >> v1:172.21.15.122:6789/0 conn(0x7fb258073570 legacy=0x7fb25806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.588+0000 7fb256ffd640 1 -- 172.21.15.122:0/184898590 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2013417198 0 0) 0x7fb258152c30 con 0x7fb258073570 2024-04-16T15:40:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.588+0000 7fb2577fe640 1 -- 172.21.15.122:0/184898590 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2581531f0 con 0x7fb25814cd40 2024-04-16T15:40:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.588+0000 7fb2577fe640 1 --2- 172.21.15.122:0/184898590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 0x7fb258073b40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2580adb90 tx=0x7fb24802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab608ab5f0e5f797 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb256ffd640 1 -- 172.21.15.122:0/184898590 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248035020 con 0x7fb25814cd40 2024-04-16T15:40:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb256ffd640 1 -- 172.21.15.122:0/184898590 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb248031870 con 0x7fb25814cd40 2024-04-16T15:40:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb256ffd640 1 -- 172.21.15.122:0/184898590 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248031c40 con 0x7fb25814cd40 2024-04-16T15:40:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb25e00d640 1 -- 172.21.15.122:0/184898590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 msgr2=0x7fb258073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb25e00d640 1 --2- 172.21.15.122:0/184898590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 0x7fb258073b40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2580adb90 tx=0x7fb24802b910 comp rx=0 tx=0).stop 2024-04-16T15:40:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb25e00d640 1 -- 172.21.15.122:0/184898590 shutdown_connections 2024-04-16T15:40:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb25e00d640 1 --2- 172.21.15.122:0/184898590 >> v2:172.21.15.71:3300/0 conn(0x7fb25806f950 0x7fb25806fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb25e00d640 1 --2- 172.21.15.122:0/184898590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 0x7fb258073b40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb25e00d640 1 -- 172.21.15.122:0/184898590 >> 172.21.15.122:0/184898590 conn(0x7fb2580c8470 msgr2=0x7fb2580c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb25e00d640 1 -- 172.21.15.122:0/184898590 shutdown_connections 2024-04-16T15:40:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.589+0000 7fb25e00d640 1 -- 172.21.15.122:0/184898590 wait complete. 2024-04-16T15:40:00.594 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:00.590+0000 7f4d70e42040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T15:40:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.591+0000 7fb25e00d640 1 Processor -- start 2024-04-16T15:40:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.591+0000 7fb25e00d640 1 -- start start 2024-04-16T15:40:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb25e00d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb25806f950 0x7fb25814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb25e00d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb258073570 0x7fb2580abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb25e00d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 0x7fb2580ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb25e00d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb25806df00 con 0x7fb25806f950 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb25e00d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb25806dd80 con 0x7fb25814cd40 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb25e00d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb258070340 con 0x7fb258073570 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb24ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb258073570 0x7fb2580abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb257fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 0x7fb2580ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb24ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb258073570 0x7fb2580abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:37036/0 (socket says 172.21.15.122:37036) 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb24ffff640 1 -- 172.21.15.122:0/197316902 learned_addr learned my addr 172.21.15.122:0/197316902 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.592+0000 7fb2577fe640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb25806f950 0x7fb25814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb257fff640 1 -- 172.21.15.122:0/197316902 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb258073570 msgr2=0x7fb2580abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb257fff640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb258073570 0x7fb2580abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb257fff640 1 -- 172.21.15.122:0/197316902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb25806f950 msgr2=0x7fb25814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb257fff640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb25806f950 0x7fb25814f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb257fff640 1 -- 172.21.15.122:0/197316902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb258147eb0 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb24ffff640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb258073570 0x7fb2580abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb257fff640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 0x7fb2580ac230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb250009d90 tx=0x7fb25000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb254ff9640 1 -- 172.21.15.122:0/197316902 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250018020 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb254ff9640 1 -- 172.21.15.122:0/197316902 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb250014480 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2580aa2a0 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb254ff9640 1 -- 172.21.15.122:0/197316902 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250014870 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2580acfe0 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb254ff9640 1 -- 172.21.15.122:0/197316902 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb250005d10 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.593+0000 7fb254ff9640 1 -- 172.21.15.122:0/197316902 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fb250013800 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.594+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb258068be0 con 0x7fb25814cd40 2024-04-16T15:40:00.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.596+0000 7fb254ff9640 1 -- 172.21.15.122:0/197316902 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb250022030 con 0x7fb25814cd40 2024-04-16T15:40:00.643 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:00.644+0000 7f4d70e42040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T15:40:00.643 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:00.643+0000 7fe1d6e37040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T15:40:00.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.668+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2eec121c-f3b5-4f74-8b1e-f389c7d99f3b", "id": 2} v 0) -- 0x7fb258072b10 con 0x7fb25814cd40 2024-04-16T15:40:00.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.672+0000 7fb254ff9640 1 -- 172.21.15.122:0/197316902 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2eec121c-f3b5-4f74-8b1e-f389c7d99f3b", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fb250010040 con 0x7fb25814cd40 2024-04-16T15:40:00.672 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-04-16T15:40:00.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 msgr2=0x7fb2580ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 0x7fb2580ac230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb250009d90 tx=0x7fb25000c490 comp rx=0 tx=0).stop 2024-04-16T15:40:00.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 shutdown_connections 2024-04-16T15:40:00.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb25814cd40 0x7fb2580ac230 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb258073570 0x7fb2580abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 --2- 172.21.15.122:0/197316902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb25806f950 0x7fb25814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 >> 172.21.15.122:0/197316902 conn(0x7fb2580c8470 msgr2=0x7fb2580c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:00.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 shutdown_connections 2024-04-16T15:40:00.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.673+0000 7fb25e00d640 1 -- 172.21.15.122:0/197316902 wait complete. 2024-04-16T15:40:00.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new f55ea5db-5519-4ae3-bcaa-1725a4eabd8c 3 2024-04-16T15:40:00.689 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:00.690+0000 7f4d70e42040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T15:40:00.691 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:00.691+0000 7fe1d6e37040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T15:40:00.744 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:00.744+0000 7f4d70e42040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T15:40:00.748 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:00.748+0000 7fe1d6e37040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T15:40:00.786 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:00.786+0000 7f4d70e42040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.787+0000 7f34559a8640 1 Processor -- start 2024-04-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.787+0000 7f34559a8640 1 -- start start 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f34559a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500c4080 0x7f34500c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f34559a8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34500c5c90 con 0x7f345006cbb0 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f34559a8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34500c73a0 con 0x7f34500a6c20 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f34559a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f34500c5a40 con 0x7f34500c4080 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f344e7fc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f34500a6c20 0x7f34500c17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:41910/0 (socket says 172.21.15.122:41910) 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f344f7fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500c4080 0x7f34500c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f344e7fc640 1 -- 172.21.15.122:0/3397243647 learned_addr learned my addr 172.21.15.122:0/3397243647 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f344f7fe640 1 -- 172.21.15.122:0/3397243647 >> v1:172.21.15.122:6789/0 conn(0x7f34500a6c20 legacy=0x7f34500c17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f344f7fe640 1 -- 172.21.15.122:0/3397243647 >> v1:172.21.15.71:6789/0 conn(0x7f345006cbb0 legacy=0x7f3450062d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.788+0000 7f344f7fe640 1 -- 172.21.15.122:0/3397243647 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34500c7680 con 0x7f34500c4080 2024-04-16T15:40:00.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.789+0000 7f344f7fe640 1 --2- 172.21.15.122:0/3397243647 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500c4080 0x7f34500c0f70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3448009dc0 tx=0x7f344802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2955440e94a101a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:00.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.789+0000 7f344dffb640 1 -- 172.21.15.122:0/3397243647 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448039070 con 0x7f34500c4080 2024-04-16T15:40:00.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.789+0000 7f344dffb640 1 -- 172.21.15.122:0/3397243647 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344802cc40 con 0x7f34500c4080 2024-04-16T15:40:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.789+0000 7f344dffb640 1 -- 172.21.15.122:0/3397243647 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448034440 con 0x7f34500c4080 2024-04-16T15:40:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.790+0000 7f34559a8640 1 -- 172.21.15.122:0/3397243647 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500c4080 msgr2=0x7f34500c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.790+0000 7f34559a8640 1 --2- 172.21.15.122:0/3397243647 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500c4080 0x7f34500c0f70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3448009dc0 tx=0x7f344802be90 comp rx=0 tx=0).stop 2024-04-16T15:40:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.790+0000 7f34559a8640 1 -- 172.21.15.122:0/3397243647 shutdown_connections 2024-04-16T15:40:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.790+0000 7f34559a8640 1 --2- 172.21.15.122:0/3397243647 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500c4080 0x7f34500c0f70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.790+0000 7f34559a8640 1 -- 172.21.15.122:0/3397243647 >> 172.21.15.122:0/3397243647 conn(0x7f34500c4640 msgr2=0x7f34500c4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.790+0000 7f34559a8640 1 -- 172.21.15.122:0/3397243647 shutdown_connections 2024-04-16T15:40:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.790+0000 7f34559a8640 1 -- 172.21.15.122:0/3397243647 wait complete. 2024-04-16T15:40:00.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.791+0000 7f34559a8640 1 Processor -- start 2024-04-16T15:40:00.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.791+0000 7f34559a8640 1 -- start start 2024-04-16T15:40:00.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f34559a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345006cbb0 0x7f3450149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f34559a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500a6c20 0x7f345006ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f34559a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34500c4080 0x7f34501570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f34559a8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3450159dc0 con 0x7f34500c4080 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f34559a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3450159ae0 con 0x7f345006cbb0 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f34559a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f345006cf90 con 0x7f34500a6c20 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345006cbb0 0x7f3450149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344e7fc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500a6c20 0x7f345006ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345006cbb0 0x7f3450149120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32834/0 (socket says 172.21.15.122:32834) 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344effd640 1 -- 172.21.15.122:0/3215134470 learned_addr learned my addr 172.21.15.122:0/3215134470 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344f7fe640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34500c4080 0x7f34501570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344effd640 1 -- 172.21.15.122:0/3215134470 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500a6c20 msgr2=0x7f345006ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344effd640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500a6c20 0x7f345006ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344effd640 1 -- 172.21.15.122:0/3215134470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34500c4080 msgr2=0x7f34501570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344effd640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34500c4080 0x7f34501570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344effd640 1 -- 172.21.15.122:0/3215134470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3450151730 con 0x7f345006cbb0 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.792+0000 7f344e7fc640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500a6c20 0x7f345006ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.793+0000 7f344effd640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345006cbb0 0x7f3450149120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f344000c430 tx=0x7f344000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.793+0000 7f3447fff640 1 -- 172.21.15.122:0/3215134470 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400052d0 con 0x7f345006cbb0 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.793+0000 7f3447fff640 1 -- 172.21.15.122:0/3215134470 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3440005490 con 0x7f345006cbb0 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.793+0000 7f3447fff640 1 -- 172.21.15.122:0/3215134470 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440007260 con 0x7f345006cbb0 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.793+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3450151a10 con 0x7f345006cbb0 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.793+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3450073ba0 con 0x7f345006cbb0 2024-04-16T15:40:00.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.793+0000 7f3447fff640 1 -- 172.21.15.122:0/3215134470 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3440007400 con 0x7f345006cbb0 2024-04-16T15:40:00.796 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:00.793+0000 7fe1d6e37040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T15:40:00.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.793+0000 7f3447fff640 1 -- 172.21.15.122:0/3215134470 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f3440011590 con 0x7f345006cbb0 2024-04-16T15:40:00.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.794+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3450074ad0 con 0x7f345006cbb0 2024-04-16T15:40:00.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.796+0000 7f3447fff640 1 -- 172.21.15.122:0/3215134470 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3440003a50 con 0x7f345006cbb0 2024-04-16T15:40:00.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.884+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f55ea5db-5519-4ae3-bcaa-1725a4eabd8c", "id": 3} v 0) -- 0x7f3450068be0 con 0x7f345006cbb0 2024-04-16T15:40:00.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.887+0000 7f3447fff640 1 -- 172.21.15.122:0/3215134470 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f55ea5db-5519-4ae3-bcaa-1725a4eabd8c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f3440003d10 con 0x7f345006cbb0 2024-04-16T15:40:00.887 INFO:teuthology.orchestra.run.smithi122.stdout:3 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345006cbb0 msgr2=0x7f3450149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345006cbb0 0x7f3450149120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f344000c430 tx=0x7f344000c8f0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 shutdown_connections 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34500c4080 0x7f34501570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34500a6c20 0x7f345006ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 --2- 172.21.15.122:0/3215134470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345006cbb0 0x7f3450149120 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 >> 172.21.15.122:0/3215134470 conn(0x7f34500c4640 msgr2=0x7f3450151f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 shutdown_connections 2024-04-16T15:40:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:00.888+0000 7f34559a8640 1 -- 172.21.15.122:0/3215134470 wait complete. 2024-04-16T15:40:00.899 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new adf7b0dd-31d3-483c-8914-b3efe320eccc 4 2024-04-16T15:40:00.910 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:00.905+0000 7f4d70e42040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T15:40:00.931 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:00.927+0000 7fe1d6e37040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T15:40:00.974 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:00.975+0000 7f4d70e42040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T15:40:01.001 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:01.001+0000 7fe1d6e37040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T15:40:01.020 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:01.021+0000 7f4d70e42040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T15:40:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.038+0000 7f7518f32640 1 Processor -- start 2024-04-16T15:40:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.038+0000 7f7518f32640 1 -- start start 2024-04-16T15:40:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f7518f32640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f75140c3fb0 0x7f75140c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f7518f32640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c7e20 0x7f751415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f7518f32640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f751414de20 con 0x7f751414da40 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f7518f32640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f75140c82d0 con 0x7f75140c3fb0 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f7518f32640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f75140c9120 con 0x7f75140c7e20 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f7512d9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c7e20 0x7f751415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f751259c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f751414da40 0x7f75140ac4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:41918/0 (socket says 172.21.15.122:41918) 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f751259c640 1 -- 172.21.15.122:0/2531913690 learned_addr learned my addr 172.21.15.122:0/2531913690 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.039+0000 7f750bfff640 1 --2- 172.21.15.122:0/2531913690 >> v2:172.21.15.71:3300/0 conn(0x7f75140c3fb0 0x7f75140c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.040+0000 7f7511d9b640 1 -- 172.21.15.122:0/2531913690 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740505092 0 0) 0x7f7504001110 con 0x7f751414da40 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.040+0000 7f7511d9b640 1 -- 172.21.15.122:0/2531913690 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7500003520 con 0x7f751414da40 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.040+0000 7f7512d9d640 1 -- 172.21.15.122:0/2531913690 >> v2:172.21.15.71:3300/0 conn(0x7f75140c3fb0 msgr2=0x7f75140c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.040+0000 7f7512d9d640 1 --2- 172.21.15.122:0/2531913690 >> v2:172.21.15.71:3300/0 conn(0x7f75140c3fb0 0x7f75140c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.040+0000 7f7512d9d640 1 -- 172.21.15.122:0/2531913690 >> v1:172.21.15.122:6789/0 conn(0x7f751414da40 legacy=0x7f75140ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.040+0000 7f7512d9d640 1 -- 172.21.15.122:0/2531913690 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75141548b0 con 0x7f75140c7e20 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.040+0000 7f7512d9d640 1 --2- 172.21.15.122:0/2531913690 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c7e20 0x7f751415e2c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f750c009dc0 tx=0x7f750c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d66fa7131eddfb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7511d9b640 1 -- 172.21.15.122:0/2531913690 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c039070 con 0x7f75140c7e20 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7511d9b640 1 -- 172.21.15.122:0/2531913690 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750c02cc40 con 0x7f75140c7e20 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7511d9b640 1 -- 172.21.15.122:0/2531913690 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c034440 con 0x7f75140c7e20 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7518f32640 1 -- 172.21.15.122:0/2531913690 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c7e20 msgr2=0x7f751415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7518f32640 1 --2- 172.21.15.122:0/2531913690 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c7e20 0x7f751415e2c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f750c009dc0 tx=0x7f750c02be90 comp rx=0 tx=0).stop 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7518f32640 1 -- 172.21.15.122:0/2531913690 shutdown_connections 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7518f32640 1 --2- 172.21.15.122:0/2531913690 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c7e20 0x7f751415e2c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7518f32640 1 --2- 172.21.15.122:0/2531913690 >> v2:172.21.15.71:3300/0 conn(0x7f75140c3fb0 0x7f75140c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7518f32640 1 -- 172.21.15.122:0/2531913690 >> 172.21.15.122:0/2531913690 conn(0x7f75140c9420 msgr2=0x7f75140c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7518f32640 1 -- 172.21.15.122:0/2531913690 shutdown_connections 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.041+0000 7f7518f32640 1 -- 172.21.15.122:0/2531913690 wait complete. 2024-04-16T15:40:01.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7518f32640 1 Processor -- start 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7518f32640 1 -- start start 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7518f32640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c3fb0 0x7f7514156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7518f32640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75140c7e20 0x7f7514150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7518f32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f751414da40 0x7f75141dd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7518f32640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75140c73c0 con 0x7f75140c7e20 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7518f32640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75141545d0 con 0x7f751414da40 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7518f32640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f75140c4a10 con 0x7f75140c3fb0 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f751259c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c3fb0 0x7f7514156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f751259c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c3fb0 0x7f7514156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:37060/0 (socket says 172.21.15.122:37060) 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f7512d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f751414da40 0x7f75141dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f751259c640 1 -- 172.21.15.122:0/4131739146 learned_addr learned my addr 172.21.15.122:0/4131739146 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.043+0000 7f750bfff640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75140c7e20 0x7f7514150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f751259c640 1 -- 172.21.15.122:0/4131739146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f751414da40 msgr2=0x7f75141dd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f751259c640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f751414da40 0x7f75141dd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f751259c640 1 -- 172.21.15.122:0/4131739146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75140c7e20 msgr2=0x7f7514150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f751259c640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75140c7e20 0x7f7514150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f751259c640 1 -- 172.21.15.122:0/4131739146 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7514155840 con 0x7f75140c3fb0 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f7512d9d640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f751414da40 0x7f75141dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f750bfff640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75140c7e20 0x7f7514150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f751259c640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c3fb0 0x7f7514156420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f750400c760 tx=0x7f750400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f750affd640 1 -- 172.21.15.122:0/4131739146 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504017070 con 0x7f75140c3fb0 2024-04-16T15:40:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f750affd640 1 -- 172.21.15.122:0/4131739146 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7504005ad0 con 0x7f75140c3fb0 2024-04-16T15:40:01.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7514155b20 con 0x7f75140c3fb0 2024-04-16T15:40:01.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f750affd640 1 -- 172.21.15.122:0/4131739146 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504007e20 con 0x7f75140c3fb0 2024-04-16T15:40:01.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f751414fe10 con 0x7f75140c3fb0 2024-04-16T15:40:01.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f750affd640 1 -- 172.21.15.122:0/4131739146 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f750401a070 con 0x7f75140c3fb0 2024-04-16T15:40:01.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.044+0000 7f750affd640 1 -- 172.21.15.122:0/4131739146 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f7504012400 con 0x7f75140c3fb0 2024-04-16T15:40:01.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.045+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75140c8e90 con 0x7f75140c3fb0 2024-04-16T15:40:01.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.047+0000 7f750affd640 1 -- 172.21.15.122:0/4131739146 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7504021070 con 0x7f75140c3fb0 2024-04-16T15:40:01.052 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:01.051+0000 7fe1d6e37040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T15:40:01.073 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:01.074+0000 7f4d70e42040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T15:40:01.106 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:01.105+0000 7fe1d6e37040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T15:40:01.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.120+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "adf7b0dd-31d3-483c-8914-b3efe320eccc", "id": 4} v 0) -- 0x7f7514068bc0 con 0x7f75140c3fb0 2024-04-16T15:40:01.121 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:01.120+0000 7f4d70e42040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T15:40:01.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.125+0000 7f750affd640 1 -- 172.21.15.122:0/4131739146 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "adf7b0dd-31d3-483c-8914-b3efe320eccc", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f751414de20 con 0x7f75140c3fb0 2024-04-16T15:40:01.127 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c3fb0 msgr2=0x7f7514156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c3fb0 0x7f7514156420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f750400c760 tx=0x7f750400cc20 comp rx=0 tx=0).stop 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 shutdown_connections 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f751414da40 0x7f75141dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75140c7e20 0x7f7514150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 --2- 172.21.15.122:0/4131739146 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f75140c3fb0 0x7f7514156420 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 >> 172.21.15.122:0/4131739146 conn(0x7f75140c9420 msgr2=0x7f75140aa670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 shutdown_connections 2024-04-16T15:40:01.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.126+0000 7f7518f32640 1 -- 172.21.15.122:0/4131739146 wait complete. 2024-04-16T15:40:01.137 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 4669c4df-9ec7-40fe-96ae-49a6b0a97324 5 2024-04-16T15:40:01.155 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:01.155+0000 7fe1d6e37040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T15:40:01.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.242+0000 7f64f999d640 1 Processor -- start 2024-04-16T15:40:01.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.242+0000 7f64f999d640 1 -- start start 2024-04-16T15:40:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64f999d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f64f4150f50 0x7f64f40c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64f999d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f40c54d0 con 0x7f64f4154090 2024-04-16T15:40:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64f999d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f40c57b0 con 0x7f64f4157830 2024-04-16T15:40:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64f999d640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f64f4157ce0 con 0x7f64f4150f50 2024-04-16T15:40:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64ebfff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f64f4157830 0x7f64f40713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:41922/0 (socket says 172.21.15.122:41922) 2024-04-16T15:40:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64ebfff640 1 -- 172.21.15.122:0/2386393142 learned_addr learned my addr 172.21.15.122:0/2386393142 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-04-16T15:40:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64f899b640 1 --2- 172.21.15.122:0/2386393142 >> v2:172.21.15.71:3300/0 conn(0x7f64f4150f50 0x7f64f40c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64eb7fe640 1 -- 172.21.15.122:0/2386393142 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 326902777 0 0) 0x7f64f40c57b0 con 0x7f64f4157830 2024-04-16T15:40:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64eb7fe640 1 -- 172.21.15.122:0/2386393142 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64dc003520 con 0x7f64f4157830 2024-04-16T15:40:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64eb7fe640 1 -- 172.21.15.122:0/2386393142 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1324658115 0 0) 0x7f64f40c54d0 con 0x7f64f4154090 2024-04-16T15:40:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64eb7fe640 1 -- 172.21.15.122:0/2386393142 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64f40c57b0 con 0x7f64f4154090 2024-04-16T15:40:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64f899b640 1 -- 172.21.15.122:0/2386393142 >> v1:172.21.15.122:6789/0 conn(0x7f64f4157830 legacy=0x7f64f40713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64f899b640 1 -- 172.21.15.122:0/2386393142 >> v1:172.21.15.71:6789/0 conn(0x7f64f4154090 legacy=0x7f64f406f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.243+0000 7f64f899b640 1 -- 172.21.15.122:0/2386393142 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f40c5a90 con 0x7f64f4150f50 2024-04-16T15:40:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.244+0000 7f64f899b640 1 --2- 172.21.15.122:0/2386393142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4150f50 0x7f64f40c71d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64f00097a0 tx=0x7f64f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6fae880a9bde55 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.244+0000 7f64eb7fe640 1 -- 172.21.15.122:0/2386393142 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0034070 con 0x7f64f4150f50 2024-04-16T15:40:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.244+0000 7f64eb7fe640 1 -- 172.21.15.122:0/2386393142 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f0030610 con 0x7f64f4150f50 2024-04-16T15:40:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.244+0000 7f64eb7fe640 1 -- 172.21.15.122:0/2386393142 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f00309e0 con 0x7f64f4150f50 2024-04-16T15:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.245+0000 7f64f999d640 1 -- 172.21.15.122:0/2386393142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4150f50 msgr2=0x7f64f40c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.245+0000 7f64f999d640 1 --2- 172.21.15.122:0/2386393142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4150f50 0x7f64f40c71d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64f00097a0 tx=0x7f64f002b960 comp rx=0 tx=0).stop 2024-04-16T15:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.245+0000 7f64f999d640 1 -- 172.21.15.122:0/2386393142 shutdown_connections 2024-04-16T15:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.245+0000 7f64f999d640 1 --2- 172.21.15.122:0/2386393142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4150f50 0x7f64f40c71d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.245+0000 7f64f999d640 1 -- 172.21.15.122:0/2386393142 >> 172.21.15.122:0/2386393142 conn(0x7f64f40ab080 msgr2=0x7f64f40ab490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.245+0000 7f64f999d640 1 -- 172.21.15.122:0/2386393142 shutdown_connections 2024-04-16T15:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.245+0000 7f64f999d640 1 -- 172.21.15.122:0/2386393142 wait complete. 2024-04-16T15:40:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.246+0000 7f64f999d640 1 Processor -- start 2024-04-16T15:40:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.246+0000 7f64f999d640 1 -- start start 2024-04-16T15:40:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f999d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f64f4150f50 0x7f64f41502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f999d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f4154090 0x7f64f4155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f999d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4157830 0x7f64f4152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f999d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f64f406fb70 con 0x7f64f4157830 2024-04-16T15:40:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f999d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4153c00 con 0x7f64f4154090 2024-04-16T15:40:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f999d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f64f406fcf0 con 0x7f64f4150f50 2024-04-16T15:40:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f64f4150f50 0x7f64f41502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f64f4150f50 0x7f64f41502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:37064/0 (socket says 172.21.15.122:37064) 2024-04-16T15:40:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 -- 172.21.15.122:0/883250895 learned_addr learned my addr 172.21.15.122:0/883250895 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64ebfff640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f4154090 0x7f64f4155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f919c640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4157830 0x7f64f4152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 -- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f4154090 msgr2=0x7f64f4155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f4154090 0x7f64f4155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 -- 172.21.15.122:0/883250895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4157830 msgr2=0x7f64f4152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4157830 0x7f64f4152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 -- 172.21.15.122:0/883250895 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f40ac380 con 0x7f64f4150f50 2024-04-16T15:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64ebfff640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f4154090 0x7f64f4155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.247+0000 7f64f899b640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f64f4150f50 0x7f64f41502e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009890 tx=0x7f64f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.248+0000 7f64e97fa640 1 -- 172.21.15.122:0/883250895 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0034040 con 0x7f64f4150f50 2024-04-16T15:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.248+0000 7f64e97fa640 1 -- 172.21.15.122:0/883250895 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f0030410 con 0x7f64f4150f50 2024-04-16T15:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.248+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f40ac660 con 0x7f64f4150f50 2024-04-16T15:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.248+0000 7f64e97fa640 1 -- 172.21.15.122:0/883250895 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0006e10 con 0x7f64f4150f50 2024-04-16T15:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.248+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f41eb850 con 0x7f64f4150f50 2024-04-16T15:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.248+0000 7f64e97fa640 1 -- 172.21.15.122:0/883250895 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f64f0037070 con 0x7f64f4150f50 2024-04-16T15:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.248+0000 7f64e97fa640 1 -- 172.21.15.122:0/883250895 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f64f00423f0 con 0x7f64f4150f50 2024-04-16T15:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.249+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f41546f0 con 0x7f64f4150f50 2024-04-16T15:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.251+0000 7f64e97fa640 1 -- 172.21.15.122:0/883250895 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f64f41546f0 con 0x7f64f4150f50 2024-04-16T15:40:01.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.324+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4669c4df-9ec7-40fe-96ae-49a6b0a97324", "id": 5} v 0) -- 0x7f64f41eba50 con 0x7f64f4150f50 2024-04-16T15:40:01.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.329+0000 7f64e97fa640 1 -- 172.21.15.122:0/883250895 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4669c4df-9ec7-40fe-96ae-49a6b0a97324", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f64f00305c0 con 0x7f64f4150f50 2024-04-16T15:40:01.330 INFO:teuthology.orchestra.run.smithi122.stdout:5 2024-04-16T15:40:01.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f64f4150f50 msgr2=0x7f64f41502e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f64f4150f50 0x7f64f41502e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009890 tx=0x7f64f00046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 shutdown_connections 2024-04-16T15:40:01.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4157830 0x7f64f4152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64f4154090 0x7f64f4155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 --2- 172.21.15.122:0/883250895 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f64f4150f50 0x7f64f41502e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 >> 172.21.15.122:0/883250895 conn(0x7f64f40ab080 msgr2=0x7f64f40c2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 shutdown_connections 2024-04-16T15:40:01.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.330+0000 7f64f999d640 1 -- 172.21.15.122:0/883250895 wait complete. 2024-04-16T15:40:01.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 8defc38b-b99c-41f3-b1af-03ac9ac9eaa9 6 2024-04-16T15:40:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.486+0000 7fb7bb715640 1 Processor -- start 2024-04-16T15:40:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.486+0000 7fb7bb715640 1 -- start start 2024-04-16T15:40:01.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7bb715640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb7b414fd70 0x7fb7b406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7bb715640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7bb715640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b40734f0 con 0x7fb7b4070500 2024-04-16T15:40:01.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7bb715640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40c3380 con 0x7fb7b414fd70 2024-04-16T15:40:01.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7bb715640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b40737d0 con 0x7fb7b40c28a0 2024-04-16T15:40:01.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7b8cb0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7b8cb0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:37068/0 (socket says 172.21.15.122:37068) 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7b8cb0640 1 -- 172.21.15.122:0/2758221606 learned_addr learned my addr 172.21.15.122:0/2758221606 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.487+0000 7fb7b94b1640 1 --2- 172.21.15.122:0/2758221606 >> v2:172.21.15.71:3300/0 conn(0x7fb7b414fd70 0x7fb7b406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.488+0000 7fb7b8cb0640 1 -- 172.21.15.122:0/2758221606 >> v2:172.21.15.71:3300/0 conn(0x7fb7b414fd70 msgr2=0x7fb7b406fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.488+0000 7fb7b8cb0640 1 --2- 172.21.15.122:0/2758221606 >> v2:172.21.15.71:3300/0 conn(0x7fb7b414fd70 0x7fb7b406fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.488+0000 7fb7b8cb0640 1 -- 172.21.15.122:0/2758221606 >> v1:172.21.15.71:6789/0 conn(0x7fb7b4070500 legacy=0x7fb7b4155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.488+0000 7fb7b8cb0640 1 -- 172.21.15.122:0/2758221606 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b40c3010 con 0x7fb7b40c28a0 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.488+0000 7fb7b8cb0640 1 --2- 172.21.15.122:0/2758221606 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406ffc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009870 tx=0x7fb7b002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2904963134637ae server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.488+0000 7fb7a7fff640 1 -- 172.21.15.122:0/2758221606 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0034070 con 0x7fb7b40c28a0 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.488+0000 7fb7a7fff640 1 -- 172.21.15.122:0/2758221606 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b0030840 con 0x7fb7b40c28a0 2024-04-16T15:40:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7a7fff640 1 -- 172.21.15.122:0/2758221606 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0030c10 con 0x7fb7b40c28a0 2024-04-16T15:40:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7bb715640 1 -- 172.21.15.122:0/2758221606 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 msgr2=0x7fb7b406ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7bb715640 1 --2- 172.21.15.122:0/2758221606 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406ffc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009870 tx=0x7fb7b002b980 comp rx=0 tx=0).stop 2024-04-16T15:40:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7bb715640 1 -- 172.21.15.122:0/2758221606 shutdown_connections 2024-04-16T15:40:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7bb715640 1 --2- 172.21.15.122:0/2758221606 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406ffc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7bb715640 1 --2- 172.21.15.122:0/2758221606 >> v2:172.21.15.71:3300/0 conn(0x7fb7b414fd70 0x7fb7b406fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7bb715640 1 -- 172.21.15.122:0/2758221606 >> 172.21.15.122:0/2758221606 conn(0x7fb7b40aa220 msgr2=0x7fb7b40aa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7bb715640 1 -- 172.21.15.122:0/2758221606 shutdown_connections 2024-04-16T15:40:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.489+0000 7fb7bb715640 1 -- 172.21.15.122:0/2758221606 wait complete. 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.490+0000 7fb7bb715640 1 Processor -- start 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.490+0000 7fb7bb715640 1 -- start start 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7bb715640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7bb715640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b414fd70 0x7fb7b4151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7bb715640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7b4147d10 0x7fb7b4148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7bb715640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40c4c60 con 0x7fb7b4147d10 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7bb715640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40c4ae0 con 0x7fb7b414fd70 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7bb715640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b41564d0 con 0x7fb7b40c28a0 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b94b1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b8cb0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b414fd70 0x7fb7b4151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b94b1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:37076/0 (socket says 172.21.15.122:37076) 2024-04-16T15:40:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b94b1640 1 -- 172.21.15.122:0/2619217363 learned_addr learned my addr 172.21.15.122:0/2619217363 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b9cb2640 1 --2- 172.21.15.122:0/2619217363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7b4147d10 0x7fb7b4148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b94b1640 1 -- 172.21.15.122:0/2619217363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b414fd70 msgr2=0x7fb7b4151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b94b1640 1 --2- 172.21.15.122:0/2619217363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b414fd70 0x7fb7b4151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b94b1640 1 -- 172.21.15.122:0/2619217363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7b4147d10 msgr2=0x7fb7b4148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b94b1640 1 --2- 172.21.15.122:0/2619217363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7b4147d10 0x7fb7b4148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.491+0000 7fb7b94b1640 1 -- 172.21.15.122:0/2619217363 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b40c22b0 con 0x7fb7b40c28a0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.492+0000 7fb7b94b1640 1 --2- 172.21.15.122:0/2619217363 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406dbb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800e870 tx=0x7fb7a800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.492+0000 7fb7a5ffb640 1 -- 172.21.15.122:0/2619217363 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8017070 con 0x7fb7b40c28a0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.492+0000 7fb7a5ffb640 1 -- 172.21.15.122:0/2619217363 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a800c940 con 0x7fb7b40c28a0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.492+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b4152350 con 0x7fb7b40c28a0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.492+0000 7fb7a5ffb640 1 -- 172.21.15.122:0/2619217363 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800cd30 con 0x7fb7b40c28a0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.492+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b40acc40 con 0x7fb7b40c28a0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.492+0000 7fb7a5ffb640 1 -- 172.21.15.122:0/2619217363 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb7a801a070 con 0x7fb7b40c28a0 2024-04-16T15:40:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.492+0000 7fb7a5ffb640 1 -- 172.21.15.122:0/2619217363 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fb7a80064d0 con 0x7fb7b40c28a0 2024-04-16T15:40:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.493+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b4068be0 con 0x7fb7b40c28a0 2024-04-16T15:40:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.495+0000 7fb7a5ffb640 1 -- 172.21.15.122:0/2619217363 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb7a8006770 con 0x7fb7b40c28a0 2024-04-16T15:40:01.504 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:01.505+0000 7f4d70e42040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T15:40:01.557 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:01.557+0000 7fe1d6e37040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T15:40:01.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.568+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8defc38b-b99c-41f3-b1af-03ac9ac9eaa9", "id": 6} v 0) -- 0x7fb7b4073c90 con 0x7fb7b40c28a0 2024-04-16T15:40:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.576+0000 7fb7a5ffb640 1 -- 172.21.15.122:0/2619217363 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8defc38b-b99c-41f3-b1af-03ac9ac9eaa9", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fb7a8005990 con 0x7fb7b40c28a0 2024-04-16T15:40:01.576 INFO:teuthology.orchestra.run.smithi122.stdout:6 2024-04-16T15:40:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 msgr2=0x7fb7b406dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 --2- 172.21.15.122:0/2619217363 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406dbb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800e870 tx=0x7fb7a800ed30 comp rx=0 tx=0).stop 2024-04-16T15:40:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 shutdown_connections 2024-04-16T15:40:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 --2- 172.21.15.122:0/2619217363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7b4147d10 0x7fb7b4148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 --2- 172.21.15.122:0/2619217363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7b414fd70 0x7fb7b4151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 --2- 172.21.15.122:0/2619217363 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7b40c28a0 0x7fb7b406dbb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 >> 172.21.15.122:0/2619217363 conn(0x7fb7b40aa220 msgr2=0x7fb7b40aa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 shutdown_connections 2024-04-16T15:40:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.577+0000 7fb7bb715640 1 -- 172.21.15.122:0/2619217363 wait complete. 2024-04-16T15:40:01.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 87c94baa-0e15-4613-93ff-4cf8d0a1db33 7 2024-04-16T15:40:01.639 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:01.641+0000 7f4d70e42040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T15:40:01.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.690+0000 7fe7ddf22640 1 Processor -- start 2024-04-16T15:40:01.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.690+0000 7fe7ddf22640 1 -- start start 2024-04-16T15:40:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7ddf22640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7d80c3280 0x7fe7d806f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7ddf22640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe7d8159bb0 0x7fe7d8152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7ddf22640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d8074060 con 0x7fe7d8072cb0 2024-04-16T15:40:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7ddf22640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d80c3a90 con 0x7fe7d8159bb0 2024-04-16T15:40:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7ddf22640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d80c1d20 con 0x7fe7d80c3280 2024-04-16T15:40:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7d6ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7d80c3280 0x7fe7d806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7d6ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7d80c3280 0x7fe7d806f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32864/0 (socket says 172.21.15.122:32864) 2024-04-16T15:40:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7d6ffd640 1 -- 172.21.15.122:0/2716891236 learned_addr learned my addr 172.21.15.122:0/2716891236 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.691+0000 7fe7d7fff640 1 --2- 172.21.15.122:0/2716891236 >> v2:172.21.15.71:3300/0 conn(0x7fe7d8159bb0 0x7fe7d8152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.692+0000 7fe7d6ffd640 1 -- 172.21.15.122:0/2716891236 >> v2:172.21.15.71:3300/0 conn(0x7fe7d8159bb0 msgr2=0x7fe7d8152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.692+0000 7fe7d6ffd640 1 --2- 172.21.15.122:0/2716891236 >> v2:172.21.15.71:3300/0 conn(0x7fe7d8159bb0 0x7fe7d8152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.692+0000 7fe7d6ffd640 1 -- 172.21.15.122:0/2716891236 >> v1:172.21.15.71:6789/0 conn(0x7fe7d8072cb0 legacy=0x7fe7d80c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.692+0000 7fe7d6ffd640 1 -- 172.21.15.122:0/2716891236 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d8159fe0 con 0x7fe7d80c3280 2024-04-16T15:40:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.692+0000 7fe7d6ffd640 1 --2- 172.21.15.122:0/2716891236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d80c3280 0x7fe7d806f980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe7d806e790 tx=0x7fe7cc02ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f4bfc86ee56ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7d67fc640 1 -- 172.21.15.122:0/2716891236 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc02f850 con 0x7fe7d80c3280 2024-04-16T15:40:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7d67fc640 1 -- 172.21.15.122:0/2716891236 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7cc02f9f0 con 0x7fe7d80c3280 2024-04-16T15:40:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7d67fc640 1 -- 172.21.15.122:0/2716891236 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc02fdc0 con 0x7fe7d80c3280 2024-04-16T15:40:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7ddf22640 1 -- 172.21.15.122:0/2716891236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d80c3280 msgr2=0x7fe7d806f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7ddf22640 1 --2- 172.21.15.122:0/2716891236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d80c3280 0x7fe7d806f980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe7d806e790 tx=0x7fe7cc02ba30 comp rx=0 tx=0).stop 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7ddf22640 1 -- 172.21.15.122:0/2716891236 shutdown_connections 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7ddf22640 1 --2- 172.21.15.122:0/2716891236 >> v2:172.21.15.71:3300/0 conn(0x7fe7d8159bb0 0x7fe7d8152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7ddf22640 1 --2- 172.21.15.122:0/2716891236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d80c3280 0x7fe7d806f980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7ddf22640 1 -- 172.21.15.122:0/2716891236 >> 172.21.15.122:0/2716891236 conn(0x7fe7d80aa220 msgr2=0x7fe7d80aa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7ddf22640 1 -- 172.21.15.122:0/2716891236 shutdown_connections 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.693+0000 7fe7ddf22640 1 -- 172.21.15.122:0/2716891236 wait complete. 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.694+0000 7fe7ddf22640 1 Processor -- start 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7ddf22640 1 -- start start 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7ddf22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8072cb0 0x7fe7d80ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7ddf22640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7d80c3280 0x7fe7d80ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7ddf22640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7d8159bb0 0x7fe7d81ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7ddf22640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8073d80 con 0x7fe7d8159bb0 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7ddf22640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d80c28c0 con 0x7fe7d8072cb0 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7ddf22640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d80c19b0 con 0x7fe7d80c3280 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8072cb0 0x7fe7d80ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7d6ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7d80c3280 0x7fe7d80ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8072cb0 0x7fe7d80ab520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32880/0 (socket says 172.21.15.122:32880) 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7d77fe640 1 -- 172.21.15.122:0/885970642 learned_addr learned my addr 172.21.15.122:0/885970642 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.695+0000 7fe7d7fff640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7d8159bb0 0x7fe7d81ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7d77fe640 1 -- 172.21.15.122:0/885970642 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7d80c3280 msgr2=0x7fe7d80ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7d77fe640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7d80c3280 0x7fe7d80ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7d77fe640 1 -- 172.21.15.122:0/885970642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7d8159bb0 msgr2=0x7fe7d81ebc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7d77fe640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7d8159bb0 0x7fe7d81ebc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7d77fe640 1 -- 172.21.15.122:0/885970642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d80c80e0 con 0x7fe7d8072cb0 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7d6ffd640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7d80c3280 0x7fe7d80ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:01.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7d77fe640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8072cb0 0x7fe7d80ab520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c400 tx=0x7fe7c800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7bffff640 1 -- 172.21.15.122:0/885970642 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80052d0 con 0x7fe7d8072cb0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7bffff640 1 -- 172.21.15.122:0/885970642 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c8005490 con 0x7fe7d8072cb0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7bffff640 1 -- 172.21.15.122:0/885970642 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8007260 con 0x7fe7d8072cb0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d81e26d0 con 0x7fe7d8072cb0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d81e2c90 con 0x7fe7d8072cb0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7bffff640 1 -- 172.21.15.122:0/885970642 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe7c8007400 con 0x7fe7d8072cb0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.696+0000 7fe7bffff640 1 -- 172.21.15.122:0/885970642 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fe7c8011bf0 con 0x7fe7d8072cb0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.697+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d8068be0 con 0x7fe7d8072cb0 2024-04-16T15:40:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.699+0000 7fe7bffff640 1 -- 172.21.15.122:0/885970642 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe7c80035f0 con 0x7fe7d8072cb0 2024-04-16T15:40:01.701 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:01.701+0000 7fe1d6e37040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T15:40:01.747 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:01.749+0000 7f4d70e42040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T15:40:01.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.772+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "87c94baa-0e15-4613-93ff-4cf8d0a1db33", "id": 7} v 0) -- 0x7fe7d81eb8a0 con 0x7fe7d8072cb0 2024-04-16T15:40:01.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.777+0000 7fe7bffff640 1 -- 172.21.15.122:0/885970642 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "87c94baa-0e15-4613-93ff-4cf8d0a1db33", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fe7c8003a00 con 0x7fe7d8072cb0 2024-04-16T15:40:01.777 INFO:teuthology.orchestra.run.smithi122.stdout:7 2024-04-16T15:40:01.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.777+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8072cb0 msgr2=0x7fe7d80ab520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.777+0000 7fe7ddf22640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8072cb0 0x7fe7d80ab520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800c400 tx=0x7fe7c800c8c0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.778+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 shutdown_connections 2024-04-16T15:40:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.778+0000 7fe7ddf22640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7d8159bb0 0x7fe7d81ebc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.778+0000 7fe7ddf22640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7d80c3280 0x7fe7d80ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.778+0000 7fe7ddf22640 1 --2- 172.21.15.122:0/885970642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8072cb0 0x7fe7d80ab520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.778+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 >> 172.21.15.122:0/885970642 conn(0x7fe7d80aa220 msgr2=0x7fe7d80c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.778+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 shutdown_connections 2024-04-16T15:40:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:01.778+0000 7fe7ddf22640 1 -- 172.21.15.122:0/885970642 wait complete. 2024-04-16T15:40:01.789 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-16T15:40:01.789 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-16T15:40:01.792 INFO:tasks.ceph.osd.0:Started 2024-04-16T15:40:01.792 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-16T15:40:01.792 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-16T15:40:01.794 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:01.795+0000 7f4d70e42040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T15:40:01.795 INFO:tasks.ceph.osd.1:Started 2024-04-16T15:40:01.796 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-16T15:40:01.796 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-16T15:40:01.798 INFO:tasks.ceph.osd.2:Started 2024-04-16T15:40:01.799 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-16T15:40:01.799 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-16T15:40:01.802 INFO:tasks.ceph.osd.3:Started 2024-04-16T15:40:01.802 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-16T15:40:01.802 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-16T15:40:01.805 INFO:tasks.ceph.osd.4:Started 2024-04-16T15:40:01.806 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-16T15:40:01.806 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-16T15:40:01.808 INFO:tasks.ceph.osd.5:Started 2024-04-16T15:40:01.808 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-16T15:40:01.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-16T15:40:01.810 INFO:tasks.ceph.osd.6:Started 2024-04-16T15:40:01.811 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-16T15:40:01.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-16T15:40:01.813 INFO:tasks.ceph.osd.7:Started 2024-04-16T15:40:01.813 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:01.826 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:01.826+0000 7fe1d6e37040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T15:40:01.879 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:01.879+0000 7fe1d6e37040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T15:40:01.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.937+0000 7f2c55647640 1 Processor -- start 2024-04-16T15:40:01.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.937+0000 7f2c55647640 1 -- start start 2024-04-16T15:40:01.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.938+0000 7f2c55647640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2c5013f250 0x7f2c500c0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.938+0000 7f2c55647640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c50156a10 con 0x7f2c500ca3e0 2024-04-16T15:40:01.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.938+0000 7f2c55647640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c500c5f80 con 0x7f2c5013ed00 2024-04-16T15:40:01.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.938+0000 7f2c55647640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f2c500cb350 con 0x7f2c5013f250 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.938+0000 7f2c4f7fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2c5013f250 0x7f2c500c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.938+0000 7f2c4effd640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f2c500ca3e0 0x7f2c500c6a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:36172/0 (socket says 172.21.15.71:36172) 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.938+0000 7f2c4effd640 1 -- 172.21.15.71:0/2447953692 learned_addr learned my addr 172.21.15.71:0/2447953692 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.939+0000 7f2c4f7fe640 1 -- 172.21.15.71:0/2447953692 >> v1:172.21.15.122:6789/0 conn(0x7f2c5013ed00 legacy=0x7f2c5015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.939+0000 7f2c4f7fe640 1 -- 172.21.15.71:0/2447953692 >> v1:172.21.15.71:6789/0 conn(0x7f2c500ca3e0 legacy=0x7f2c500c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.939+0000 7f2c4f7fe640 1 -- 172.21.15.71:0/2447953692 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c500c6260 con 0x7f2c5013f250 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.939+0000 7f2c4f7fe640 1 --2- 172.21.15.71:0/2447953692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c500c0370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009d90 tx=0x7f2c4802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c128c77d33e1f7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c4dffb640 1 -- 172.21.15.71:0/2447953692 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48039070 con 0x7f2c5013f250 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c4dffb640 1 -- 172.21.15.71:0/2447953692 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4802cbd0 con 0x7f2c5013f250 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c4dffb640 1 -- 172.21.15.71:0/2447953692 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48034440 con 0x7f2c5013f250 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c55647640 1 -- 172.21.15.71:0/2447953692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 msgr2=0x7f2c500c0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c55647640 1 --2- 172.21.15.71:0/2447953692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c500c0370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009d90 tx=0x7f2c4802be20 comp rx=0 tx=0).stop 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c55647640 1 -- 172.21.15.71:0/2447953692 shutdown_connections 2024-04-16T15:40:01.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c55647640 1 --2- 172.21.15.71:0/2447953692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c500c0370 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c55647640 1 -- 172.21.15.71:0/2447953692 >> 172.21.15.71:0/2447953692 conn(0x7f2c500ad100 msgr2=0x7f2c500ad510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c55647640 1 -- 172.21.15.71:0/2447953692 shutdown_connections 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.940+0000 7f2c55647640 1 -- 172.21.15.71:0/2447953692 wait complete. 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.942+0000 7f2c55647640 1 Processor -- start 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.942+0000 7f2c55647640 1 -- start start 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.942+0000 7f2c55647640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c500ca3e0 0x7f2c501efd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.942+0000 7f2c55647640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5013ed00 0x7f2c5013b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.942+0000 7f2c55647640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c50137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.942+0000 7f2c55647640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50156cf0 con 0x7f2c5013f250 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.942+0000 7f2c55647640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501567c0 con 0x7f2c5013ed00 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c55647640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2c500c65d0 con 0x7f2c500ca3e0 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c50137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c50137ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43122/0 (socket says 172.21.15.71:43122) 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 -- 172.21.15.71:0/3449133430 learned_addr learned my addr 172.21.15.71:0/3449133430 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5013ed00 0x7f2c5013b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4effd640 1 --2- 172.21.15.71:0/3449133430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c500ca3e0 0x7f2c501efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 -- 172.21.15.71:0/3449133430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c500ca3e0 msgr2=0x7f2c501efd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 --2- 172.21.15.71:0/3449133430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c500ca3e0 0x7f2c501efd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 -- 172.21.15.71:0/3449133430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5013ed00 msgr2=0x7f2c5013b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 --2- 172.21.15.71:0/3449133430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5013ed00 0x7f2c5013b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 -- 172.21.15.71:0/3449133430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c501511a0 con 0x7f2c5013f250 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c4f7fe640 1 --2- 172.21.15.71:0/3449133430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c50137ef0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009eb0 tx=0x7f2c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c477fe640 1 -- 172.21.15.71:0/3449133430 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48039050 con 0x7f2c5013f250 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c477fe640 1 -- 172.21.15.71:0/3449133430 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c48045070 con 0x7f2c5013f250 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c477fe640 1 -- 172.21.15.71:0/3449133430 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4803ea70 con 0x7f2c5013f250 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c50155ae0 con 0x7f2c5013f250 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.943+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c50151410 con 0x7f2c5013f250 2024-04-16T15:40:01.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.944+0000 7f2c477fe640 1 -- 172.21.15.71:0/3449133430 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c4802ca50 con 0x7f2c5013f250 2024-04-16T15:40:01.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.944+0000 7f2c477fe640 1 -- 172.21.15.71:0/3449133430 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2c4803ec10 con 0x7f2c5013f250 2024-04-16T15:40:01.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.944+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c50068cf0 con 0x7f2c5013f250 2024-04-16T15:40:01.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:01.948+0000 7f2c477fe640 1 -- 172.21.15.71:0/3449133430 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2c4804a460 con 0x7f2c5013f250 2024-04-16T15:40:01.949 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:01.951+0000 7f4d70e42040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T15:40:02.005 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:02.005+0000 7fe1d6e37040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T15:40:02.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.008+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c500cb050 con 0x7f2c5013f250 2024-04-16T15:40:02.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.009+0000 7f2c477fe640 1 -- 172.21.15.71:0/3449133430 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2c4804a640 con 0x7f2c5013f250 2024-04-16T15:40:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:02.008 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":10,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:01.774843+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 msgr2=0x7f2c50137ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 --2- 172.21.15.71:0/3449133430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c50137ef0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009eb0 tx=0x7f2c48004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 shutdown_connections 2024-04-16T15:40:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 --2- 172.21.15.71:0/3449133430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c5013f250 0x7f2c50137ef0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 --2- 172.21.15.71:0/3449133430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5013ed00 0x7f2c5013b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 --2- 172.21.15.71:0/3449133430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c500ca3e0 0x7f2c501efd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 >> 172.21.15.71:0/3449133430 conn(0x7f2c500ad100 msgr2=0x7f2c5013d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 shutdown_connections 2024-04-16T15:40:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:02.010+0000 7f2c55647640 1 -- 172.21.15.71:0/3449133430 wait complete. 2024-04-16T15:40:02.019 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-16T15:40:02.020 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-16T15:40:02.048 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:02.050+0000 7f4d70e42040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T15:40:02.098 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:02.098+0000 7fe1d6e37040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T15:40:02.205 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:02.206+0000 7f4d70e42040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T15:40:02.247 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:02.248+0000 7f4d70e42040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T15:40:02.253 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:02.253+0000 7fe1d6e37040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T15:40:02.295 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:02.295+0000 7fe1d6e37040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T15:40:02.400 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:02.402+0000 7f4d70e42040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T15:40:02.411 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-04-16T15:40:02.411+0000 7fb46da0f640 -1 Falling back to public interface 2024-04-16T15:40:02.420 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-04-16T15:40:02.420+0000 7eff36d2a640 -1 Falling back to public interface 2024-04-16T15:40:02.422 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-04-16T15:40:02.423+0000 7f8073821640 -1 Falling back to public interface 2024-04-16T15:40:02.429 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-04-16T15:40:02.430+0000 7f228a214640 -1 Falling back to public interface 2024-04-16T15:40:02.430 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-04-16T15:40:02.431+0000 7fa269019640 -1 Falling back to public interface 2024-04-16T15:40:02.434 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-04-16T15:40:02.435+0000 7f5b71bdf640 -1 Falling back to public interface 2024-04-16T15:40:02.436 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-04-16T15:40:02.436+0000 7f334f017640 -1 Falling back to public interface 2024-04-16T15:40:02.437 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-04-16T15:40:02.436+0000 7f8766d81640 -1 Falling back to public interface 2024-04-16T15:40:02.437 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-04-16T15:40:02.437+0000 7f8765d30640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-04-16T15:40:02.451 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:02.452+0000 7fe1d6e37040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T15:40:02.824 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:02.916 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.918+0000 7efdb9cc5640 1 Processor -- start 2024-04-16T15:40:02.916 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.918+0000 7efdb9cc5640 1 -- start start 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.918+0000 7efdb9cc5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40c72a0 0x7efdb41566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.918+0000 7efdb9cc5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7efdb4152e10 0x7efdb4153250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.918+0000 7efdb9cc5640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb40af930 con 0x7efdb40aec60 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.918+0000 7efdb9cc5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7efdb4156c00 con 0x7efdb4152e10 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.918+0000 7efdb9cc5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efdb40afbd0 con 0x7efdb40c72a0 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb3fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7efdb40aec60 0x7efdb40af0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:36180/0 (socket says 172.21.15.71:36180) 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb3fff640 1 -- 172.21.15.71:0/267575956 learned_addr learned my addr 172.21.15.71:0/267575956 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb2ffd640 1 --2- 172.21.15.71:0/267575956 >> v2:172.21.15.122:3300/0 conn(0x7efdb4152e10 0x7efdb4153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb37fe640 1 --2- 172.21.15.71:0/267575956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40c72a0 0x7efdb41566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb27fc640 1 -- 172.21.15.71:0/267575956 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409038862 0 0) 0x7efdb40af930 con 0x7efdb40aec60 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb27fc640 1 -- 172.21.15.71:0/267575956 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd9c003520 con 0x7efdb40aec60 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb37fe640 1 -- 172.21.15.71:0/267575956 >> v2:172.21.15.122:3300/0 conn(0x7efdb4152e10 msgr2=0x7efdb4153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:02.919 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb37fe640 1 --2- 172.21.15.71:0/267575956 >> v2:172.21.15.122:3300/0 conn(0x7efdb4152e10 0x7efdb4153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.920 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb37fe640 1 -- 172.21.15.71:0/267575956 >> v1:172.21.15.71:6789/0 conn(0x7efdb40aec60 legacy=0x7efdb40af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:02.920 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb37fe640 1 -- 172.21.15.71:0/267575956 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb40afeb0 con 0x7efdb40c72a0 2024-04-16T15:40:02.920 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.919+0000 7efdb2ffd640 1 --2- 172.21.15.71:0/267575956 >> v2:172.21.15.122:3300/0 conn(0x7efdb4152e10 0x7efdb4153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:02.920 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.920+0000 7efdb37fe640 1 --2- 172.21.15.71:0/267575956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40c72a0 0x7efdb41566c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efda40097a0 tx=0x7efda402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f9dec4173c0c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:02.920 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.920+0000 7efdb27fc640 1 -- 172.21.15.71:0/267575956 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4035020 con 0x7efdb40c72a0 2024-04-16T15:40:02.920 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.920+0000 7efdb27fc640 1 -- 172.21.15.71:0/267575956 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4031830 con 0x7efdb40c72a0 2024-04-16T15:40:02.920 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.920+0000 7efdb27fc640 1 -- 172.21.15.71:0/267575956 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4031c00 con 0x7efdb40c72a0 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.920+0000 7efdb9cc5640 1 -- 172.21.15.71:0/267575956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40c72a0 msgr2=0x7efdb41566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.921+0000 7efdb9cc5640 1 --2- 172.21.15.71:0/267575956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40c72a0 0x7efdb41566c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efda40097a0 tx=0x7efda402b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.921+0000 7efdb9cc5640 1 -- 172.21.15.71:0/267575956 shutdown_connections 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.921+0000 7efdb9cc5640 1 --2- 172.21.15.71:0/267575956 >> v2:172.21.15.122:3300/0 conn(0x7efdb4152e10 0x7efdb4153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.921+0000 7efdb9cc5640 1 --2- 172.21.15.71:0/267575956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40c72a0 0x7efdb41566c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.921+0000 7efdb9cc5640 1 -- 172.21.15.71:0/267575956 >> 172.21.15.71:0/267575956 conn(0x7efdb40cb790 msgr2=0x7efdb40cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.921+0000 7efdb9cc5640 1 -- 172.21.15.71:0/267575956 shutdown_connections 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.921+0000 7efdb9cc5640 1 -- 172.21.15.71:0/267575956 wait complete. 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.922+0000 7efdb9cc5640 1 Processor -- start 2024-04-16T15:40:02.923 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.922+0000 7efdb9cc5640 1 -- start start 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb9cc5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40aec60 0x7efdb4159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb9cc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb40c72a0 0x7efdb414a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb9cc5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdb4152e10 0x7efdb414adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb9cc5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40ae740 con 0x7efdb40c72a0 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb9cc5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40ae460 con 0x7efdb4152e10 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb9cc5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efdb40c9800 con 0x7efdb40aec60 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb40c72a0 0x7efdb414a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb40c72a0 0x7efdb414a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43130/0 (socket says 172.21.15.71:43130) 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 -- 172.21.15.71:0/1384216715 learned_addr learned my addr 172.21.15.71:0/1384216715 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb37fe640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40aec60 0x7efdb4159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb3fff640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdb4152e10 0x7efdb414adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 -- 172.21.15.71:0/1384216715 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40aec60 msgr2=0x7efdb4159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40aec60 0x7efdb4159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.924 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 -- 172.21.15.71:0/1384216715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdb4152e10 msgr2=0x7efdb414adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdb4152e10 0x7efdb414adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 -- 172.21.15.71:0/1384216715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb40c32e0 con 0x7efdb40c72a0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb2ffd640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb40c72a0 0x7efdb414a8b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efda800ec10 tx=0x7efda800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efd9bfff640 1 -- 172.21.15.71:0/1384216715 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8017070 con 0x7efdb40c72a0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efd9bfff640 1 -- 172.21.15.71:0/1384216715 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda800cc30 con 0x7efdb40c72a0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb41eb540 con 0x7efdb40c72a0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efd9bfff640 1 -- 172.21.15.71:0/1384216715 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8013470 con 0x7efdb40c72a0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb41ebb00 con 0x7efdb40c72a0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efd9bfff640 1 -- 172.21.15.71:0/1384216715 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efda801a070 con 0x7efdb40c72a0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.923+0000 7efd9bfff640 1 -- 172.21.15.71:0/1384216715 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7efda801c4f0 con 0x7efdb40c72a0 2024-04-16T15:40:02.925 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.924+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb4075cb0 con 0x7efdb40c72a0 2024-04-16T15:40:02.926 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.926+0000 7efd9bfff640 1 -- 172.21.15.71:0/1384216715 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efda8013610 con 0x7efdb40c72a0 2024-04-16T15:40:02.987 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.988+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdb40c6e10 con 0x7efdb40c72a0 2024-04-16T15:40:02.987 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.989+0000 7efd9bfff640 1 -- 172.21.15.71:0/1384216715 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7efda80137f0 con 0x7efdb40c72a0 2024-04-16T15:40:02.988 INFO:teuthology.misc.health.smithi071.stdout: 2024-04-16T15:40:02.988 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":10,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:01.774843+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.989+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb40c72a0 msgr2=0x7efdb414a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.989+0000 7efdb9cc5640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb40c72a0 0x7efdb414a8b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efda800ec10 tx=0x7efda800c520 comp rx=0 tx=0).stop 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.990+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 shutdown_connections 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.990+0000 7efdb9cc5640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdb4152e10 0x7efdb414adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.990+0000 7efdb9cc5640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdb40c72a0 0x7efdb414a8b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.990+0000 7efdb9cc5640 1 --2- 172.21.15.71:0/1384216715 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efdb40aec60 0x7efdb4159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.990+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 >> 172.21.15.71:0/1384216715 conn(0x7efdb40cb790 msgr2=0x7efdb4155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.990+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 shutdown_connections 2024-04-16T15:40:02.989 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:02.990+0000 7efdb9cc5640 1 -- 172.21.15.71:0/1384216715 wait complete. 2024-04-16T15:40:02.998 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-16T15:40:03.237 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:03.238+0000 7f4d70e42040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T15:40:03.288 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:03.289+0000 7fe1d6e37040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T15:40:03.514 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:03.516+0000 7f4d70e42040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T15:40:03.566 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:03.567+0000 7fe1d6e37040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T15:40:03.568 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:03.569+0000 7f4d70e42040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T15:40:03.617 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:03.618+0000 7fe1d6e37040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T15:40:03.682 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:03.684+0000 7f4d70e42040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T15:40:03.738 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-04-16T15:40:03.739+0000 7fe1d6e37040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T15:40:04.698 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:04.699+0000 7f4d12255640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-16T15:40:04.698 INFO:tasks.ceph.mgr.y.smithi071.stderr:2024-04-16T15:40:04.699+0000 7f4d12255640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-16T15:40:05.898 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-04-16T15:40:05.899+0000 7f8073821640 -1 osd.0 0 log_to_monitors true 2024-04-16T15:40:05.906 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-04-16T15:40:05.907+0000 7fb46da0f640 -1 osd.7 0 log_to_monitors true 2024-04-16T15:40:05.911 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-04-16T15:40:05.912+0000 7eff36d2a640 -1 osd.6 0 log_to_monitors true 2024-04-16T15:40:05.956 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-04-16T15:40:05.957+0000 7f5b71bdf640 -1 osd.3 0 log_to_monitors true 2024-04-16T15:40:05.961 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-04-16T15:40:05.963+0000 7f228a214640 -1 osd.1 0 log_to_monitors true 2024-04-16T15:40:05.961 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-04-16T15:40:05.963+0000 7fa269019640 -1 osd.2 0 log_to_monitors true 2024-04-16T15:40:05.962 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-04-16T15:40:05.963+0000 7f8766d81640 -1 osd.5 0 log_to_monitors true 2024-04-16T15:40:05.968 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-04-16T15:40:05.969+0000 7f334f017640 -1 osd.4 0 log_to_monitors true 2024-04-16T15:40:09.738 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-04-16T15:40:09.738+0000 7eff32cc0640 -1 osd.6 0 waiting for initial osdmap 2024-04-16T15:40:09.746 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-04-16T15:40:09.747+0000 7fb4693c0640 -1 osd.7 0 waiting for initial osdmap 2024-04-16T15:40:09.751 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-04-16T15:40:09.753+0000 7eff2aae0640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T15:40:09.757 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-04-16T15:40:09.759+0000 7fb4611e0640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T15:40:09.779 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-04-16T15:40:09.780+0000 7f8762d17640 -1 osd.5 0 waiting for initial osdmap 2024-04-16T15:40:09.793 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-04-16T15:40:09.795+0000 7f875ab37640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T15:40:09.805 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:09.854 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-04-16T15:40:09.852+0000 7fa264a7d640 -1 osd.2 0 waiting for initial osdmap 2024-04-16T15:40:09.865 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-04-16T15:40:09.866+0000 7f5b6db75640 -1 osd.3 0 waiting for initial osdmap 2024-04-16T15:40:09.871 INFO:tasks.ceph.osd.2.smithi071.stderr:2024-04-16T15:40:09.872+0000 7fa25c89d640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T15:40:09.875 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-04-16T15:40:09.877+0000 7f2285b61640 -1 osd.1 0 waiting for initial osdmap 2024-04-16T15:40:09.879 INFO:tasks.ceph.osd.3.smithi071.stderr:2024-04-16T15:40:09.881+0000 7f5b65995640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T15:40:09.888 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-04-16T15:40:09.889+0000 7f806f1f7640 -1 osd.0 0 waiting for initial osdmap 2024-04-16T15:40:09.889 INFO:tasks.ceph.osd.1.smithi071.stderr:2024-04-16T15:40:09.891+0000 7f227d981640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T15:40:09.901 INFO:tasks.ceph.osd.0.smithi071.stderr:2024-04-16T15:40:09.903+0000 7f8067017640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T15:40:09.939 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.941+0000 7f98b44c5640 1 Processor -- start 2024-04-16T15:40:09.940 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.941+0000 7f98b44c5640 1 -- start start 2024-04-16T15:40:09.941 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b44c5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac14dd60 0x7f98ac0c4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:09.941 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b44c5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f98ac0ad690 0x7f98ac0c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:09.941 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b44c5640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ac0c4750 con 0x7f98ac154200 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b44c5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f98ac14e140 con 0x7f98ac0ad690 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b44c5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac0c4a30 con 0x7f98ac14dd60 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b2a62640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f98ac154200 0x7f98ac073450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:45524/0 (socket says 172.21.15.71:45524) 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b2a62640 1 -- 172.21.15.71:0/1840202576 learned_addr learned my addr 172.21.15.71:0/1840202576 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b1a60640 1 --2- 172.21.15.71:0/1840202576 >> v2:172.21.15.122:3300/0 conn(0x7f98ac0ad690 0x7f98ac0c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b2261640 1 --2- 172.21.15.71:0/1840202576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac14dd60 0x7f98ac0c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b125f640 1 -- 172.21.15.71:0/1840202576 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998301244 0 0) 0x7f98ac0c4750 con 0x7f98ac154200 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b125f640 1 -- 172.21.15.71:0/1840202576 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9898003520 con 0x7f98ac154200 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b2261640 1 -- 172.21.15.71:0/1840202576 >> v2:172.21.15.122:3300/0 conn(0x7f98ac0ad690 msgr2=0x7f98ac0c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b2261640 1 --2- 172.21.15.71:0/1840202576 >> v2:172.21.15.122:3300/0 conn(0x7f98ac0ad690 0x7f98ac0c2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b2261640 1 -- 172.21.15.71:0/1840202576 >> v1:172.21.15.71:6789/0 conn(0x7f98ac154200 legacy=0x7f98ac073450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:09.942 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b2261640 1 -- 172.21.15.71:0/1840202576 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac153990 con 0x7f98ac14dd60 2024-04-16T15:40:09.943 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.942+0000 7f98b1a60640 1 --2- 172.21.15.71:0/1840202576 >> v2:172.21.15.122:3300/0 conn(0x7f98ac0ad690 0x7f98ac0c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:09.947 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.943+0000 7f98b2261640 1 --2- 172.21.15.71:0/1840202576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac14dd60 0x7f98ac0c4040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f98a80097a0 tx=0x7f98a802b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7f773226d754d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:09.947 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.943+0000 7f98b125f640 1 -- 172.21.15.71:0/1840202576 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8038070 con 0x7f98ac14dd60 2024-04-16T15:40:09.947 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.943+0000 7f98b125f640 1 -- 172.21.15.71:0/1840202576 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a8031700 con 0x7f98ac14dd60 2024-04-16T15:40:09.947 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.943+0000 7f98b125f640 1 -- 172.21.15.71:0/1840202576 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8031ab0 con 0x7f98ac14dd60 2024-04-16T15:40:09.947 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.944+0000 7f98b44c5640 1 -- 172.21.15.71:0/1840202576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac14dd60 msgr2=0x7f98ac0c4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:09.947 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.944+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1840202576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac14dd60 0x7f98ac0c4040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f98a80097a0 tx=0x7f98a802b7a0 comp rx=0 tx=0).stop 2024-04-16T15:40:09.947 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.944+0000 7f98b44c5640 1 -- 172.21.15.71:0/1840202576 shutdown_connections 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.944+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1840202576 >> v2:172.21.15.122:3300/0 conn(0x7f98ac0ad690 0x7f98ac0c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.944+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1840202576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac14dd60 0x7f98ac0c4040 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.944+0000 7f98b44c5640 1 -- 172.21.15.71:0/1840202576 >> 172.21.15.71:0/1840202576 conn(0x7f98ac072930 msgr2=0x7f98ac072d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.944+0000 7f98b44c5640 1 -- 172.21.15.71:0/1840202576 shutdown_connections 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.944+0000 7f98b44c5640 1 -- 172.21.15.71:0/1840202576 wait complete. 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.945+0000 7f98b44c5640 1 Processor -- start 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.945+0000 7f98b44c5640 1 -- start start 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b44c5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ac0ad690 0x7f98ac0bd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b44c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98ac14dd60 0x7f98ac157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b44c5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac154200 0x7f98ac157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:09.948 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b44c5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac073ce0 con 0x7f98ac0ad690 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b44c5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac073b60 con 0x7f98ac14dd60 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b44c5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac1546b0 con 0x7f98ac154200 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ac0ad690 0x7f98ac0bd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ac0ad690 0x7f98ac0bd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50476/0 (socket says 172.21.15.71:50476) 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 -- 172.21.15.71:0/1350215100 learned_addr learned my addr 172.21.15.71:0/1350215100 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b1a60640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98ac14dd60 0x7f98ac157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2a62640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac154200 0x7f98ac157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 -- 172.21.15.71:0/1350215100 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac154200 msgr2=0x7f98ac157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac154200 0x7f98ac157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 -- 172.21.15.71:0/1350215100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98ac14dd60 msgr2=0x7f98ac157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98ac14dd60 0x7f98ac157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 -- 172.21.15.71:0/1350215100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac1ebe20 con 0x7f98ac0ad690 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b2261640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ac0ad690 0x7f98ac0bd4f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000c00 tx=0x7f98a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98a2ffd640 1 -- 172.21.15.71:0/1350215100 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8043070 con 0x7f98ac0ad690 2024-04-16T15:40:09.949 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98a2ffd640 1 -- 172.21.15.71:0/1350215100 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a803f870 con 0x7f98ac0ad690 2024-04-16T15:40:09.950 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ac156ae0 con 0x7f98ac0ad690 2024-04-16T15:40:09.950 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98a2ffd640 1 -- 172.21.15.71:0/1350215100 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8038040 con 0x7f98ac0ad690 2024-04-16T15:40:09.950 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.946+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ac1ebb40 con 0x7f98ac0ad690 2024-04-16T15:40:09.950 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.947+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ac075cb0 con 0x7f98ac0ad690 2024-04-16T15:40:09.950 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.948+0000 7f98a2ffd640 1 -- 172.21.15.71:0/1350215100 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f98a8046070 con 0x7f98ac0ad690 2024-04-16T15:40:09.950 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.948+0000 7f98a2ffd640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f989006c860 0x7f989006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:09.950 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.948+0000 7f98a2ffd640 1 -- 172.21.15.71:0/1350215100 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f98a80ac790 con 0x7f98ac0ad690 2024-04-16T15:40:09.950 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.948+0000 7f98b1a60640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f989006c860 0x7f989006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:09.951 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.949+0000 7f98b1a60640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f989006c860 0x7f989006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f989c009560 tx=0x7f989c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:09.951 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:09.953+0000 7f98a2ffd640 1 -- 172.21.15.71:0/1350215100 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98a8030070 con 0x7f98ac0ad690 2024-04-16T15:40:09.988 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-04-16T15:40:09.989+0000 7f334aa7d640 -1 osd.4 0 waiting for initial osdmap 2024-04-16T15:40:10.003 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-04-16T15:40:10.004+0000 7f334289d640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T15:40:10.128 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.129+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98ac14d340 con 0x7f98ac0ad690 2024-04-16T15:40:10.128 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.130+0000 7f98a2ffd640 1 -- 172.21.15.71:0/1350215100 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f98a807b470 con 0x7f98ac0ad690 2024-04-16T15:40:10.128 INFO:teuthology.misc.health.smithi071.stdout: 2024-04-16T15:40:10.129 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":12,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:07.705268+0000","last_up_change":"0.000000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f989006c860 msgr2=0x7f989006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f989006c860 0x7f989006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f989c009560 tx=0x7f989c009190 comp rx=0 tx=0).stop 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ac0ad690 msgr2=0x7f98ac0bd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ac0ad690 0x7f98ac0bd4f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000c00 tx=0x7f98a80046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 shutdown_connections 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f989006c860 0x7f989006ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98ac154200 0x7f98ac157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98ac14dd60 0x7f98ac157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 --2- 172.21.15.71:0/1350215100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ac0ad690 0x7f98ac0bd4f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 >> 172.21.15.71:0/1350215100 conn(0x7f98ac072930 msgr2=0x7f98ac0c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 shutdown_connections 2024-04-16T15:40:10.131 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:10.132+0000 7f98b44c5640 1 -- 172.21.15.71:0/1350215100 wait complete. 2024-04-16T15:40:10.140 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-16T15:40:16.947 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:17.049 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.050+0000 7f72e0c7b640 1 Processor -- start 2024-04-16T15:40:17.049 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.050+0000 7f72e0c7b640 1 -- start start 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72e0c7b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f72dc1568a0 0x7f72dc06ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72e0c7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc06fcb0 0x7f72dc159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72e0c7b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f72dc070190 0x7f72dc159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72e0c7b640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f72dc0727c0 con 0x7f72dc1568a0 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72e0c7b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f72dc155e20 con 0x7f72dc070190 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72e0c7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc0729f0 con 0x7f72dc06fcb0 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72da59c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f72dc1568a0 0x7f72dc06ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72da59c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f72dc1568a0 0x7f72dc06ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50504/0 (socket says 172.21.15.71:50504) 2024-04-16T15:40:17.052 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72da59c640 1 -- 172.21.15.71:0/2882864267 learned_addr learned my addr 172.21.15.71:0/2882864267 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72da59c640 1 -- 172.21.15.71:0/2882864267 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc06fcb0 msgr2=0x7f72dc159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72dad9d640 1 --2- 172.21.15.71:0/2882864267 >> v2:172.21.15.122:3300/0 conn(0x7f72dc070190 0x7f72dc159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72d1d9b640 1 --2- 172.21.15.71:0/2882864267 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc06fcb0 0x7f72dc159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72da59c640 1 --2- 172.21.15.71:0/2882864267 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc06fcb0 0x7f72dc159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72da59c640 1 -- 172.21.15.71:0/2882864267 >> v2:172.21.15.122:3300/0 conn(0x7f72dc070190 msgr2=0x7f72dc159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72da59c640 1 --2- 172.21.15.71:0/2882864267 >> v2:172.21.15.122:3300/0 conn(0x7f72dc070190 0x7f72dc159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.051+0000 7f72da59c640 1 -- 172.21.15.71:0/2882864267 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc072cd0 con 0x7f72dc1568a0 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.052+0000 7f72da59c640 1 --2- 172.21.15.71:0/2882864267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc1568a0 0x7f72dc06ca30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f72dc06eec0 tx=0x7f72cc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d8464f7a82c275 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.052+0000 7f72d9d9b640 1 -- 172.21.15.71:0/2882864267 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc02e070 con 0x7f72dc1568a0 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.052+0000 7f72d9d9b640 1 -- 172.21.15.71:0/2882864267 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc037070 con 0x7f72dc1568a0 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.052+0000 7f72d9d9b640 1 -- 172.21.15.71:0/2882864267 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc031a00 con 0x7f72dc1568a0 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2882864267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc1568a0 msgr2=0x7f72dc06ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2882864267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc1568a0 0x7f72dc06ca30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f72dc06eec0 tx=0x7f72cc02b880 comp rx=0 tx=0).stop 2024-04-16T15:40:17.053 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2882864267 shutdown_connections 2024-04-16T15:40:17.054 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2882864267 >> v2:172.21.15.122:3300/0 conn(0x7f72dc070190 0x7f72dc159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.054 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2882864267 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc06fcb0 0x7f72dc159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.054 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2882864267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc1568a0 0x7f72dc06ca30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.054 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2882864267 >> 172.21.15.71:0/2882864267 conn(0x7f72dc0c4120 msgr2=0x7f72dc0c4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:17.054 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2882864267 shutdown_connections 2024-04-16T15:40:17.054 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.053+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2882864267 wait complete. 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72e0c7b640 1 Processor -- start 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72e0c7b640 1 -- start start 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72e0c7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72dc06fcb0 0x7f72dc1ec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72e0c7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc070190 0x7f72dc157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72e0c7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc1568a0 0x7f72dc0ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72e0c7b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc15a1f0 con 0x7f72dc070190 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72e0c7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc073290 con 0x7f72dc06fcb0 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72e0c7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc156c80 con 0x7f72dc1568a0 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72d1d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc070190 0x7f72dc157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72d1d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc070190 0x7f72dc157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50518/0 (socket says 172.21.15.71:50518) 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.055+0000 7f72d1d9b640 1 -- 172.21.15.71:0/2767380589 learned_addr learned my addr 172.21.15.71:0/2767380589 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72dad9d640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc1568a0 0x7f72dc0ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.057 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72da59c640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72dc06fcb0 0x7f72dc1ec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d1d9b640 1 -- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc1568a0 msgr2=0x7f72dc0ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d1d9b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc1568a0 0x7f72dc0ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d1d9b640 1 -- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72dc06fcb0 msgr2=0x7f72dc1ec4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d1d9b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72dc06fcb0 0x7f72dc1ec4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d1d9b640 1 -- 172.21.15.71:0/2767380589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc1eba30 con 0x7f72dc070190 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72dad9d640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc1568a0 0x7f72dc0ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72da59c640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72dc06fcb0 0x7f72dc1ec4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d1d9b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc070190 0x7f72dc157190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c540 tx=0x7f72c400ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d37fe640 1 -- 172.21.15.71:0/2767380589 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4015070 con 0x7f72dc070190 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d37fe640 1 -- 172.21.15.71:0/2767380589 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c4011800 con 0x7f72dc070190 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72d37fe640 1 -- 172.21.15.71:0/2767380589 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4011bb0 con 0x7f72dc070190 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72dc0ad5c0 con 0x7f72dc070190 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.056+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72dc0aa8d0 con 0x7f72dc070190 2024-04-16T15:40:17.058 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.057+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72dc075cb0 con 0x7f72dc070190 2024-04-16T15:40:17.059 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.057+0000 7f72d37fe640 1 -- 172.21.15.71:0/2767380589 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72c4018070 con 0x7f72dc070190 2024-04-16T15:40:17.059 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.058+0000 7f72d37fe640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f72bc06c920 0x7f72bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.059 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.058+0000 7f72d37fe640 1 -- 172.21.15.71:0/2767380589 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f72c408b210 con 0x7f72dc070190 2024-04-16T15:40:17.059 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.058+0000 7f72da59c640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f72bc06c920 0x7f72bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.059 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.058+0000 7f72da59c640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f72bc06c920 0x7f72bc06ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72cc036000 tx=0x7f72cc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.061 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.062+0000 7f72d37fe640 1 -- 172.21.15.71:0/2767380589 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72c40599a0 con 0x7f72dc070190 2024-04-16T15:40:17.237 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.239+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72dc06ef90 con 0x7f72dc070190 2024-04-16T15:40:17.238 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.240+0000 7f72d37fe640 1 -- 172.21.15.71:0/2767380589 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7f72c40592a0 con 0x7f72dc070190 2024-04-16T15:40:17.239 INFO:teuthology.misc.health.smithi071.stdout: 2024-04-16T15:40:17.239 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":16,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:13.860214+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f72bc06c920 msgr2=0x7f72bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f72bc06c920 0x7f72bc06ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72cc036000 tx=0x7f72cc008030 comp rx=0 tx=0).stop 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc070190 msgr2=0x7f72dc157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc070190 0x7f72dc157190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c540 tx=0x7f72c400ca00 comp rx=0 tx=0).stop 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 shutdown_connections 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f72bc06c920 0x7f72bc06ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72dc1568a0 0x7f72dc0ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72dc070190 0x7f72dc157190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 --2- 172.21.15.71:0/2767380589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72dc06fcb0 0x7f72dc1ec4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 >> 172.21.15.71:0/2767380589 conn(0x7f72dc0c4120 msgr2=0x7f72dc1553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 shutdown_connections 2024-04-16T15:40:17.241 INFO:teuthology.misc.health.smithi071.stderr:2024-04-16T15:40:17.242+0000 7f72e0c7b640 1 -- 172.21.15.71:0/2767380589 wait complete. 2024-04-16T15:40:17.250 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-16T15:40:17.251 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-16T15:40:17.251 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-16T15:40:17.251 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-16T15:40:17.254 INFO:tasks.ceph.mds.a:Started 2024-04-16T15:40:17.254 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-16T15:40:17.254 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-16T15:40:17.256 INFO:tasks.ceph.mds.c:Started 2024-04-16T15:40:17.257 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-16T15:40:17.257 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-16T15:40:17.259 INFO:tasks.ceph.mds.b:Started 2024-04-16T15:40:17.259 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-16T15:40:17.260 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-16T15:40:17.262 INFO:tasks.ceph.mds.d:Started 2024-04-16T15:40:17.262 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-16T15:40:17.262 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:17.331 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-04-16T15:40:17.335 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-04-16T15:40:17.352 INFO:tasks.ceph.mds.b.smithi122.stdout:starting mds.b at 2024-04-16T15:40:17.353 INFO:tasks.ceph.mds.d.smithi122.stdout:starting mds.d at 2024-04-16T15:40:17.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.372+0000 7fb3ede7b640 1 Processor -- start 2024-04-16T15:40:17.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.372+0000 7fb3ede7b640 1 -- start start 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3ede7b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb3e81531a0 0x7fb3e8159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3ede7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c26c0 0x7fb3e80c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3ede7b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e80c4680 con 0x7fb3e80c1c90 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3ede7b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e80c38f0 con 0x7fb3e81531a0 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3ede7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e8071110 con 0x7fb3e80c26c0 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e77fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb3e81531a0 0x7fb3e8159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e77fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb3e81531a0 0x7fb3e8159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:33686/0 (socket says 172.21.15.71:33686) 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3deffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c26c0 0x7fb3e80c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e7fff640 1 --1- 172.21.15.71:0/1158833310 >> v1:172.21.15.122:6789/0 conn(0x7fb3e80c1c90 0x7fb3e806f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:36160/0 (socket says 172.21.15.71:36160) 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e77fe640 1 -- 172.21.15.71:0/1158833310 learned_addr learned my addr 172.21.15.71:0/1158833310 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:17.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e77fe640 1 -- 172.21.15.71:0/1158833310 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c26c0 msgr2=0x7fb3e80c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e77fe640 1 --2- 172.21.15.71:0/1158833310 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c26c0 0x7fb3e80c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e77fe640 1 -- 172.21.15.71:0/1158833310 >> v1:172.21.15.122:6789/0 conn(0x7fb3e80c1c90 legacy=0x7fb3e806f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e6ffd640 1 -- 172.21.15.71:0/1158833310 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 958172562 0 0) 0x7fb3e80c4680 con 0x7fb3e80c1c90 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3e77fe640 1 -- 172.21.15.71:0/1158833310 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e80c4960 con 0x7fb3e81531a0 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.373+0000 7fb3deffd640 1 --2- 172.21.15.71:0/1158833310 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c26c0 0x7fb3e80c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.374+0000 7fb3e77fe640 1 --2- 172.21.15.71:0/1158833310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e81531a0 0x7fb3e8159b60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009870 tx=0x7fb3d802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8babcd76c9cf6290 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.374+0000 7fb3e6ffd640 1 -- 172.21.15.71:0/1158833310 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8034070 con 0x7fb3e81531a0 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.374+0000 7fb3e6ffd640 1 -- 172.21.15.71:0/1158833310 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d8030840 con 0x7fb3e81531a0 2024-04-16T15:40:17.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.374+0000 7fb3e6ffd640 1 -- 172.21.15.71:0/1158833310 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8030c10 con 0x7fb3e81531a0 2024-04-16T15:40:17.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.375+0000 7fb3ede7b640 1 -- 172.21.15.71:0/1158833310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e81531a0 msgr2=0x7fb3e8159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.375+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/1158833310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e81531a0 0x7fb3e8159b60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009870 tx=0x7fb3d802b980 comp rx=0 tx=0).stop 2024-04-16T15:40:17.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.375+0000 7fb3ede7b640 1 -- 172.21.15.71:0/1158833310 shutdown_connections 2024-04-16T15:40:17.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.375+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/1158833310 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c26c0 0x7fb3e80c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.375+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/1158833310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e81531a0 0x7fb3e8159b60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.375+0000 7fb3ede7b640 1 -- 172.21.15.71:0/1158833310 >> 172.21.15.71:0/1158833310 conn(0x7fb3e80c4e20 msgr2=0x7fb3e80c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:17.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.375+0000 7fb3ede7b640 1 -- 172.21.15.71:0/1158833310 shutdown_connections 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.375+0000 7fb3ede7b640 1 -- 172.21.15.71:0/1158833310 wait complete. 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.376+0000 7fb3ede7b640 1 Processor -- start 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.376+0000 7fb3ede7b640 1 -- start start 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3ede7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c1c90 0x7fb3e81598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3ede7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e80c26c0 0x7fb3e814dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3ede7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3e81531a0 0x7fb3e814ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3ede7b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e806ff70 con 0x7fb3e81531a0 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3ede7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e8153580 con 0x7fb3e80c26c0 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3ede7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e80700f0 con 0x7fb3e80c1c90 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3e81531a0 0x7fb3e814ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3deffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e80c26c0 0x7fb3e814dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e77fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c1c90 0x7fb3e81598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3e81531a0 0x7fb3e814ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50562/0 (socket says 172.21.15.71:50562) 2024-04-16T15:40:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3deffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e80c26c0 0x7fb3e814dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:33700/0 (socket says 172.21.15.71:33700) 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e7fff640 1 -- 172.21.15.71:0/132448581 learned_addr learned my addr 172.21.15.71:0/132448581 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e77fe640 1 -- 172.21.15.71:0/132448581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e80c26c0 msgr2=0x7fb3e814dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e77fe640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e80c26c0 0x7fb3e814dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e77fe640 1 -- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3e81531a0 msgr2=0x7fb3e814ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e77fe640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3e81531a0 0x7fb3e814ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e77fe640 1 -- 172.21.15.71:0/132448581 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e8159eb0 con 0x7fb3e80c1c90 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3deffd640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e80c26c0 0x7fb3e814dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.377+0000 7fb3e7fff640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3e81531a0 0x7fb3e814ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.378+0000 7fb3e77fe640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c1c90 0x7fb3e81598d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8033000 tx=0x7fb3d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.378+0000 7fb3e4ff9640 1 -- 172.21.15.71:0/132448581 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8034040 con 0x7fb3e80c1c90 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.378+0000 7fb3e4ff9640 1 -- 172.21.15.71:0/132448581 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d803ab10 con 0x7fb3e80c1c90 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.378+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e8156270 con 0x7fb3e80c1c90 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.378+0000 7fb3e4ff9640 1 -- 172.21.15.71:0/132448581 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8004030 con 0x7fb3e80c1c90 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.378+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e8156830 con 0x7fb3e80c1c90 2024-04-16T15:40:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.379+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e8068cf0 con 0x7fb3e80c1c90 2024-04-16T15:40:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.379+0000 7fb3e4ff9640 1 -- 172.21.15.71:0/132448581 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3d804b060 con 0x7fb3e80c1c90 2024-04-16T15:40:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.379+0000 7fb3e4ff9640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.380+0000 7fb3e4ff9640 1 -- 172.21.15.71:0/132448581 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb3d80ad4a0 con 0x7fb3e80c1c90 2024-04-16T15:40:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.380+0000 7fb3deffd640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.380+0000 7fb3deffd640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb3c806c8f0 0x7fb3c806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb3e814f4c0 tx=0x7fb3d0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.384+0000 7fb3e4ff9640 1 -- 172.21.15.71:0/132448581 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3d803f070 con 0x7fb3e80c1c90 2024-04-16T15:40:17.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.565+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3e806fab0 con 0x7fb3e80c1c90 2024-04-16T15:40:17.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.566+0000 7fb3e4ff9640 1 -- 172.21.15.71:0/132448581 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7fb3d807b530 con 0x7fb3e80c1c90 2024-04-16T15:40:17.565 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:17.565 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":16,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:13.860214+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:17.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb3c806c8f0 msgr2=0x7fb3c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb3c806c8f0 0x7fb3c806edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb3e814f4c0 tx=0x7fb3d0009190 comp rx=0 tx=0).stop 2024-04-16T15:40:17.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c1c90 msgr2=0x7fb3e81598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c1c90 0x7fb3e81598d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8033000 tx=0x7fb3d80046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 shutdown_connections 2024-04-16T15:40:17.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb3c806c8f0 0x7fb3c806edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3e81531a0 0x7fb3e814ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3e80c26c0 0x7fb3e814dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 --2- 172.21.15.71:0/132448581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3e80c1c90 0x7fb3e81598d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 >> 172.21.15.71:0/132448581 conn(0x7fb3e80c4e20 msgr2=0x7fb3e80c5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:17.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 shutdown_connections 2024-04-16T15:40:17.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.568+0000 7fb3ede7b640 1 -- 172.21.15.71:0/132448581 wait complete. 2024-04-16T15:40:17.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T15:40:17.578 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.685+0000 7f768bdba640 1 Processor -- start 2024-04-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.685+0000 7f768bdba640 1 -- start start 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768bdba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f76841531c0 0x7f76840c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768bdba640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f76840c93c0 0x7f7684156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768bdba640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76841535a0 con 0x7f76840beb50 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768bdba640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f76840cb7b0 con 0x7f76840c93c0 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768bdba640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7684156790 con 0x7f76841531c0 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768a357640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f76840c93c0 0x7f7684156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768a357640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f76840c93c0 0x7f7684156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45608/0 (socket says 172.21.15.71:45608) 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768a357640 1 -- 172.21.15.71:0/3407332576 learned_addr learned my addr 172.21.15.71:0/3407332576 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768a357640 1 -- 172.21.15.71:0/3407332576 >> v2:172.21.15.122:3300/0 conn(0x7f76841531c0 msgr2=0x7f76840c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f7689b56640 1 --2- 172.21.15.71:0/3407332576 >> v2:172.21.15.122:3300/0 conn(0x7f76841531c0 0x7f76840c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768a357640 1 --2- 172.21.15.71:0/3407332576 >> v2:172.21.15.122:3300/0 conn(0x7f76841531c0 0x7f76840c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768a357640 1 -- 172.21.15.71:0/3407332576 >> v1:172.21.15.122:6789/0 conn(0x7f76840beb50 legacy=0x7f7684155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.686+0000 7f768a357640 1 -- 172.21.15.71:0/3407332576 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7684159be0 con 0x7f76840c93c0 2024-04-16T15:40:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.687+0000 7f768a357640 1 --2- 172.21.15.71:0/3407332576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684156250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f768406c830 tx=0x7f767402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3334f8b7e896c503 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.687+0000 7f7688b54640 1 -- 172.21.15.71:0/3407332576 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674038070 con 0x7f76840c93c0 2024-04-16T15:40:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.687+0000 7f7688b54640 1 -- 172.21.15.71:0/3407332576 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767402f050 con 0x7f76840c93c0 2024-04-16T15:40:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.687+0000 7f7688b54640 1 -- 172.21.15.71:0/3407332576 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674031970 con 0x7f76840c93c0 2024-04-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.688+0000 7f768bdba640 1 -- 172.21.15.71:0/3407332576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 msgr2=0x7f7684156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.688+0000 7f768bdba640 1 --2- 172.21.15.71:0/3407332576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684156250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f768406c830 tx=0x7f767402b7f0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.688+0000 7f768bdba640 1 -- 172.21.15.71:0/3407332576 shutdown_connections 2024-04-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.688+0000 7f768bdba640 1 --2- 172.21.15.71:0/3407332576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684156250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.688+0000 7f768bdba640 1 --2- 172.21.15.71:0/3407332576 >> v2:172.21.15.122:3300/0 conn(0x7f76841531c0 0x7f76840c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.688+0000 7f768bdba640 1 -- 172.21.15.71:0/3407332576 >> 172.21.15.71:0/3407332576 conn(0x7f768406bb10 msgr2=0x7f768406bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.688+0000 7f768bdba640 1 -- 172.21.15.71:0/3407332576 shutdown_connections 2024-04-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.688+0000 7f768bdba640 1 -- 172.21.15.71:0/3407332576 wait complete. 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.689+0000 7f768bdba640 1 Processor -- start 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.689+0000 7f768bdba640 1 -- start start 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768bdba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76840beb50 0x7f768414c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768bdba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768bdba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76841531c0 0x7f76840beff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768bdba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f76840cbce0 con 0x7f76840c93c0 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768bdba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76840cba00 con 0x7f76840beb50 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768bdba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f768406e650 con 0x7f76841531c0 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45624/0 (socket says 172.21.15.71:45624) 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 -- 172.21.15.71:0/3997988381 learned_addr learned my addr 172.21.15.71:0/3997988381 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:17.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689b56640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76840beb50 0x7f768414c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768a357640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76841531c0 0x7f76840beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 -- 172.21.15.71:0/3997988381 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76841531c0 msgr2=0x7f76840beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76841531c0 0x7f76840beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 -- 172.21.15.71:0/3997988381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76840beb50 msgr2=0x7f768414c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76840beb50 0x7f768414c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 -- 172.21.15.71:0/3997988381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76841587c0 con 0x7f76840c93c0 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f7689355640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684154a00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f767800c780 tx=0x7f767800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f76827fc640 1 -- 172.21.15.71:0/3997988381 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678016020 con 0x7f76840c93c0 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f76827fc640 1 -- 172.21.15.71:0/3997988381 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767800aa50 con 0x7f76840c93c0 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76841570d0 con 0x7f76840c93c0 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f76827fc640 1 -- 172.21.15.71:0/3997988381 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678012420 con 0x7f76840c93c0 2024-04-16T15:40:17.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.690+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7684159230 con 0x7f76840c93c0 2024-04-16T15:40:17.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.691+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7684068be0 con 0x7f76840c93c0 2024-04-16T15:40:17.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.692+0000 7f76827fc640 1 -- 172.21.15.71:0/3997988381 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7678019020 con 0x7f76840c93c0 2024-04-16T15:40:17.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.692+0000 7f76827fc640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f767006c8f0 0x7f767006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.692+0000 7f76827fc640 1 -- 172.21.15.71:0/3997988381 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f767808b720 con 0x7f76840c93c0 2024-04-16T15:40:17.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.692+0000 7f7689b56640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f767006c8f0 0x7f767006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.693+0000 7f7689b56640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f767006c8f0 0x7f767006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f767c00aa20 tx=0x7f767c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.697+0000 7f76827fc640 1 -- 172.21.15.71:0/3997988381 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7678020030 con 0x7f76840c93c0 2024-04-16T15:40:17.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.870+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76840c9940 con 0x7f76840c93c0 2024-04-16T15:40:17.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.870+0000 7f76827fc640 1 -- 172.21.15.71:0/3997988381 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f7678059860 con 0x7f76840c93c0 2024-04-16T15:40:17.869 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f767006c8f0 msgr2=0x7f767006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f767006c8f0 0x7f767006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f767c00aa20 tx=0x7f767c008040 comp rx=0 tx=0).stop 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 msgr2=0x7f7684154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684154a00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f767800c780 tx=0x7f767800cc40 comp rx=0 tx=0).stop 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 shutdown_connections 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f767006c8f0 0x7f767006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76841531c0 0x7f76840beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76840c93c0 0x7f7684154a00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 --2- 172.21.15.71:0/3997988381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76840beb50 0x7f768414c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 >> 172.21.15.71:0/3997988381 conn(0x7f768406bb10 msgr2=0x7f76840c7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:17.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 shutdown_connections 2024-04-16T15:40:17.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.872+0000 7f768bdba640 1 -- 172.21.15.71:0/3997988381 wait complete. 2024-04-16T15:40:17.881 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:40:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.985+0000 7fe5cfcf2640 1 Processor -- start 2024-04-16T15:40:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.985+0000 7fe5cfcf2640 1 -- start start 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cfcf2640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe5c8155dc0 0x7fe5c80c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cfcf2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe5c806fac0 0x7fe5c81561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cfcf2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c80ab290 0x7fe5c81527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cfcf2640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c8073f20 con 0x7fe5c8155dc0 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cfcf2640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c8156ab0 con 0x7fe5c806fac0 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cfcf2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c80c27c0 con 0x7fe5c80ab290 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cda8e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe5c8155dc0 0x7fe5c80c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cda8e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe5c8155dc0 0x7fe5c80c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45638/0 (socket says 172.21.15.71:45638) 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.986+0000 7fe5cda8e640 1 -- 172.21.15.71:0/3654326110 learned_addr learned my addr 172.21.15.71:0/3654326110 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.987+0000 7fe5cda8e640 1 -- 172.21.15.71:0/3654326110 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c80ab290 msgr2=0x7fe5c81527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.987+0000 7fe5ce28f640 1 --2- 172.21.15.71:0/3654326110 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c80ab290 0x7fe5c81527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.987+0000 7fe5cd28d640 1 --2- 172.21.15.71:0/3654326110 >> v2:172.21.15.122:3300/0 conn(0x7fe5c806fac0 0x7fe5c81561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.987+0000 7fe5cda8e640 1 --2- 172.21.15.71:0/3654326110 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c80ab290 0x7fe5c81527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.987+0000 7fe5cda8e640 1 -- 172.21.15.71:0/3654326110 >> v2:172.21.15.122:3300/0 conn(0x7fe5c806fac0 msgr2=0x7fe5c81561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.987+0000 7fe5cda8e640 1 --2- 172.21.15.71:0/3654326110 >> v2:172.21.15.122:3300/0 conn(0x7fe5c806fac0 0x7fe5c81561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.987+0000 7fe5cda8e640 1 -- 172.21.15.71:0/3654326110 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c80c2aa0 con 0x7fe5c8155dc0 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.987+0000 7fe5cda8e640 1 --2- 172.21.15.71:0/3654326110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8155dc0 0x7fe5c80c32a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe5c806d2d0 tx=0x7fe5b802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9cbaafc2ff2e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cca8c640 1 -- 172.21.15.71:0/3654326110 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b802ea80 con 0x7fe5c8155dc0 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cca8c640 1 -- 172.21.15.71:0/3654326110 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b802ec20 con 0x7fe5c8155dc0 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cca8c640 1 -- 172.21.15.71:0/3654326110 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8009600 con 0x7fe5c8155dc0 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3654326110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8155dc0 msgr2=0x7fe5c80c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3654326110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8155dc0 0x7fe5c80c32a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe5c806d2d0 tx=0x7fe5b802e0e0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3654326110 shutdown_connections 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3654326110 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c80ab290 0x7fe5c81527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3654326110 >> v2:172.21.15.122:3300/0 conn(0x7fe5c806fac0 0x7fe5c81561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3654326110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8155dc0 0x7fe5c80c32a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3654326110 >> 172.21.15.71:0/3654326110 conn(0x7fe5c80c3e10 msgr2=0x7fe5c80c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:17.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3654326110 shutdown_connections 2024-04-16T15:40:17.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.988+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3654326110 wait complete. 2024-04-16T15:40:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.989+0000 7fe5cfcf2640 1 Processor -- start 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.989+0000 7fe5cfcf2640 1 -- start start 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cfcf2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c80ab290 0x7fe5c8142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cfcf2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c8155dc0 0x7fe5c8141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cfcf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5c81e7810 0x7fe5c81420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cfcf2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c81566e0 con 0x7fe5c80ab290 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cfcf2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80c18b0 con 0x7fe5c81e7810 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cfcf2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c8156860 con 0x7fe5c8155dc0 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cda8e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c80ab290 0x7fe5c8142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cda8e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c80ab290 0x7fe5c8142b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45644/0 (socket says 172.21.15.71:45644) 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cda8e640 1 -- 172.21.15.71:0/3045545427 learned_addr learned my addr 172.21.15.71:0/3045545427 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cd28d640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c8155dc0 0x7fe5c8141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5ce28f640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5c81e7810 0x7fe5c81420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cda8e640 1 -- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c8155dc0 msgr2=0x7fe5c8141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cda8e640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c8155dc0 0x7fe5c8141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cda8e640 1 -- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5c81e7810 msgr2=0x7fe5c81420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cda8e640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5c81e7810 0x7fe5c81420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cda8e640 1 -- 172.21.15.71:0/3045545427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c81eafd0 con 0x7fe5c80ab290 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.990+0000 7fe5cd28d640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c8155dc0 0x7fe5c8141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.991+0000 7fe5cda8e640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c80ab290 0x7fe5c8142b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8000f80 tx=0x7fe5b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.991+0000 7fe5c67fc640 1 -- 172.21.15.71:0/3045545427 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8045020 con 0x7fe5c80ab290 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.991+0000 7fe5c67fc640 1 -- 172.21.15.71:0/3045545427 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b8040660 con 0x7fe5c80ab290 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.991+0000 7fe5c67fc640 1 -- 172.21.15.71:0/3045545427 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b802ea80 con 0x7fe5c80ab290 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.991+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c81eb2b0 con 0x7fe5c80ab290 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.991+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c81eb870 con 0x7fe5c80ab290 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.992+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c8068d20 con 0x7fe5c80ab290 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.992+0000 7fe5c67fc640 1 -- 172.21.15.71:0/3045545427 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5b8048020 con 0x7fe5c80ab290 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.993+0000 7fe5c67fc640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe5b406c920 0x7fe5b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:17.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.993+0000 7fe5c67fc640 1 -- 172.21.15.71:0/3045545427 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe5b80b02d0 con 0x7fe5c80ab290 2024-04-16T15:40:17.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.993+0000 7fe5cd28d640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe5b406c920 0x7fe5b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:17.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.993+0000 7fe5cd28d640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe5b406c920 0x7fe5b406ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe5c81dda90 tx=0x7fe5bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:17.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:17.997+0000 7fe5c67fc640 1 -- 172.21.15.71:0/3045545427 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5b804f030 con 0x7fe5c80ab290 2024-04-16T15:40:18.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.208+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5c814cc50 con 0x7fe5c80ab290 2024-04-16T15:40:18.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.208+0000 7fe5c67fc640 1 -- 172.21.15.71:0/3045545427 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fe5b807ea60 con 0x7fe5c80ab290 2024-04-16T15:40:18.207 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:18.207 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T15:40:18.207 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 2 2024-04-16T15:40:18.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe5b406c920 msgr2=0x7fe5b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:18.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe5b406c920 0x7fe5b406ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe5c81dda90 tx=0x7fe5bc008040 comp rx=0 tx=0).stop 2024-04-16T15:40:18.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c80ab290 msgr2=0x7fe5c8142b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:18.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c80ab290 0x7fe5c8142b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8000f80 tx=0x7fe5b8004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:18.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 shutdown_connections 2024-04-16T15:40:18.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe5b406c920 0x7fe5b406ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5c81e7810 0x7fe5c81420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe5c8155dc0 0x7fe5c8141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 --2- 172.21.15.71:0/3045545427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c80ab290 0x7fe5c8142b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 >> 172.21.15.71:0/3045545427 conn(0x7fe5c80c3e10 msgr2=0x7fe5c80c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 shutdown_connections 2024-04-16T15:40:18.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:18.210+0000 7fe5cfcf2640 1 -- 172.21.15.71:0/3045545427 wait complete. 2024-04-16T15:40:20.221 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:40:20.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.338+0000 7f05ae088640 1 Processor -- start 2024-04-16T15:40:20.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.338+0000 7f05ae088640 1 -- start start 2024-04-16T15:40:20.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.338+0000 7f05ae088640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f05a8150f40 0x7f05a80accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05ae088640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a8153ba0 con 0x7f05a80ad210 2024-04-16T15:40:20.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05ae088640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a8153900 con 0x7f05a80ab950 2024-04-16T15:40:20.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05ae088640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f05a80ac4d0 con 0x7f05a8150f40 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f059ffff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f05a80ad210 0x7f05a80aa3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:58612/0 (socket says 172.21.15.71:58612) 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f059ffff640 1 -- 172.21.15.71:0/2012899324 learned_addr learned my addr 172.21.15.71:0/2012899324 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a77fe640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f05a8150f40 0x7f05a80accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3027734913 0 0) 0x7f05a8153ba0 con 0x7f05a80ad210 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0594003520 con 0x7f05a80ad210 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 210872456 0 0) 0x7f0594003520 con 0x7f05a80ad210 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f05a8150f40 msgr2=0x7f05a80accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f05a8150f40 0x7f05a80accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 >> v1:172.21.15.122:6789/0 conn(0x7f05a80ab950 legacy=0x7f05a815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a80ac7b0 con 0x7f05a80ad210 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f0590001670 con 0x7f05a80ad210 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 >> v1:172.21.15.71:6789/0 conn(0x7f05a80ad210 legacy=0x7f05a80aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f05a8150f40 0x7f05940039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0594003f60 0x7f0594004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0594004d40 0x7f05940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f05a80ac4d0 con 0x7f0594003f60 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f059400bb30 con 0x7f0594004d40 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f059400bd30 con 0x7f05a8150f40 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f059ffff640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0594003f60 0x7f0594004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a77fe640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f05a8150f40 0x7f05940039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.339+0000 7f05a7fff640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0594004d40 0x7f05940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.340+0000 7f059ffff640 1 -- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f05a8150f40 msgr2=0x7f05940039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.340+0000 7f059ffff640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f05a8150f40 0x7f05940039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.340+0000 7f059ffff640 1 -- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0594004d40 msgr2=0x7f05940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.340+0000 7f059ffff640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0594004d40 0x7f05940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.340+0000 7f059ffff640 1 -- 172.21.15.71:0/2012899324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0594003520 con 0x7f0594003f60 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.340+0000 7f059ffff640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0594003f60 0x7f0594004800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f059000c240 tx=0x7f0594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4472645e526e69fa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590007cf0 con 0x7f0594003f60 2024-04-16T15:40:20.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0590030070 con 0x7f0594003f60 2024-04-16T15:40:20.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05a6ffd640 1 -- 172.21.15.71:0/2012899324 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590005d30 con 0x7f0594003f60 2024-04-16T15:40:20.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05ae088640 1 -- 172.21.15.71:0/2012899324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0594003f60 msgr2=0x7f0594004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05ae088640 1 --2- 172.21.15.71:0/2012899324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0594003f60 0x7f0594004800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f059000c240 tx=0x7f0594005830 comp rx=0 tx=0).stop 2024-04-16T15:40:20.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05a7fff640 1 -- 172.21.15.71:0/2012899324 reap_dead start 2024-04-16T15:40:20.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05ae088640 1 -- 172.21.15.71:0/2012899324 shutdown_connections 2024-04-16T15:40:20.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05ae088640 1 -- 172.21.15.71:0/2012899324 >> 172.21.15.71:0/2012899324 conn(0x7f05a80c3e10 msgr2=0x7f05a80c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:20.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05ae088640 1 -- 172.21.15.71:0/2012899324 shutdown_connections 2024-04-16T15:40:20.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.341+0000 7f05ae088640 1 -- 172.21.15.71:0/2012899324 wait complete. 2024-04-16T15:40:20.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.342+0000 7f05ae088640 1 Processor -- start 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05ae088640 1 -- start start 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05ae088640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0594003f60 0x7f05a81eba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05ae088640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a81ebf80 0x7f05a80c0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05ae088640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05a814c740 0x7f05a81eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05ae088640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f05a80aac50 con 0x7f05a81ebf80 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05ae088640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05a80aaad0 con 0x7f05a814c740 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05ae088640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f05a806fdd0 con 0x7f0594003f60 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f059ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a81ebf80 0x7f05a80c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f059ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a81ebf80 0x7f05a80c0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45660/0 (socket says 172.21.15.71:45660) 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f059ffff640 1 -- 172.21.15.71:0/2525147258 learned_addr learned my addr 172.21.15.71:0/2525147258 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05a77fe640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0594003f60 0x7f05a81eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.343+0000 7f05a7fff640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05a814c740 0x7f05a81eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f059ffff640 1 -- 172.21.15.71:0/2525147258 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0594003f60 msgr2=0x7f05a81eba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f059ffff640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0594003f60 0x7f05a81eba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f059ffff640 1 -- 172.21.15.71:0/2525147258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05a814c740 msgr2=0x7f05a81eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f059ffff640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05a814c740 0x7f05a81eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f059ffff640 1 -- 172.21.15.71:0/2525147258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a8145d10 con 0x7f05a81ebf80 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f059ffff640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a81ebf80 0x7f05a80c0740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f059000c360 tx=0x7f05a80ac980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f05a4ff9640 1 -- 172.21.15.71:0/2525147258 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590001c30 con 0x7f05a81ebf80 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f05a4ff9640 1 -- 172.21.15.71:0/2525147258 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0590004bf0 con 0x7f05a81ebf80 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a81eb260 con 0x7f05a81ebf80 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f05a4ff9640 1 -- 172.21.15.71:0/2525147258 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590007070 con 0x7f05a81ebf80 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.344+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a8148990 con 0x7f05a81ebf80 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.345+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a8068d20 con 0x7f05a81ebf80 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.345+0000 7f05a4ff9640 1 -- 172.21.15.71:0/2525147258 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0590047020 con 0x7f05a81ebf80 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.346+0000 7f05a4ff9640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f058806c920 0x7f058806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.346+0000 7f05a4ff9640 1 -- 172.21.15.71:0/2525147258 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f05900ac760 con 0x7f05a81ebf80 2024-04-16T15:40:20.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.346+0000 7f05a77fe640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f058806c920 0x7f058806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.346+0000 7f05a77fe640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f058806c920 0x7f058806ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0598009bc0 tx=0x7f0598009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.351+0000 7f05a4ff9640 1 -- 172.21.15.71:0/2525147258 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f059007b030 con 0x7f05a81ebf80 2024-04-16T15:40:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.566+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05a806d680 con 0x7f05a81ebf80 2024-04-16T15:40:20.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.566+0000 7f05a4ff9640 1 -- 172.21.15.71:0/2525147258 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f059007a930 con 0x7f05a81ebf80 2024-04-16T15:40:20.565 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:20.565 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T15:40:20.566 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 3 2024-04-16T15:40:20.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f058806c920 msgr2=0x7f058806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f058806c920 0x7f058806ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0598009bc0 tx=0x7f0598009190 comp rx=0 tx=0).stop 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a81ebf80 msgr2=0x7f05a80c0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a81ebf80 0x7f05a80c0740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f059000c360 tx=0x7f05a80ac980 comp rx=0 tx=0).stop 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 shutdown_connections 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f058806c920 0x7f058806ede0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05a814c740 0x7f05a81eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a81ebf80 0x7f05a80c0740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 --2- 172.21.15.71:0/2525147258 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0594003f60 0x7f05a81eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 >> 172.21.15.71:0/2525147258 conn(0x7f05a80c3e10 msgr2=0x7f05a81502a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 shutdown_connections 2024-04-16T15:40:20.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.568+0000 7f05ae088640 1 -- 172.21.15.71:0/2525147258 wait complete. 2024-04-16T15:40:20.578 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:20.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.682+0000 7f0e47495640 1 Processor -- start 2024-04-16T15:40:20.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.682+0000 7f0e47495640 1 -- start start 2024-04-16T15:40:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e47495640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0e40151060 0x7f0e40154380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e47495640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e4006fc60 0x7f0e400c4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e47495640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0e400c4960 0x7f0e400c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e47495640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f0e40154990 con 0x7f0e400c4960 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e47495640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f0e40071520 con 0x7f0e40151060 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e47495640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0e40071750 con 0x7f0e4006fc60 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e45a32640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0e400c4960 0x7f0e400c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e45a32640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0e400c4960 0x7f0e400c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45672/0 (socket says 172.21.15.71:45672) 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e45a32640 1 -- 172.21.15.71:0/908685932 learned_addr learned my addr 172.21.15.71:0/908685932 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.683+0000 7f0e44a30640 1 --2- 172.21.15.71:0/908685932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e4006fc60 0x7f0e400c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e45231640 1 --2- 172.21.15.71:0/908685932 >> v2:172.21.15.122:3300/0 conn(0x7f0e40151060 0x7f0e40154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e45a32640 1 -- 172.21.15.71:0/908685932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e4006fc60 msgr2=0x7f0e400c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e45a32640 1 --2- 172.21.15.71:0/908685932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e4006fc60 0x7f0e400c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e45a32640 1 -- 172.21.15.71:0/908685932 >> v2:172.21.15.122:3300/0 conn(0x7f0e40151060 msgr2=0x7f0e40154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e45a32640 1 --2- 172.21.15.71:0/908685932 >> v2:172.21.15.122:3300/0 conn(0x7f0e40151060 0x7f0e40154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e45a32640 1 -- 172.21.15.71:0/908685932 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e400719f0 con 0x7f0e400c4960 2024-04-16T15:40:20.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e44a30640 1 --2- 172.21.15.71:0/908685932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e4006fc60 0x7f0e400c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e45a32640 1 --2- 172.21.15.71:0/908685932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e400c4960 0x7f0e400c72e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e400c9f90 tx=0x7f0e3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fffddbcee80466 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e37fff640 1 -- 172.21.15.71:0/908685932 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38035020 con 0x7f0e400c4960 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e37fff640 1 -- 172.21.15.71:0/908685932 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e38031870 con 0x7f0e400c4960 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.684+0000 7f0e37fff640 1 -- 172.21.15.71:0/908685932 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38031c40 con 0x7f0e400c4960 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 -- 172.21.15.71:0/908685932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e400c4960 msgr2=0x7f0e400c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 --2- 172.21.15.71:0/908685932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e400c4960 0x7f0e400c72e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e400c9f90 tx=0x7f0e3802b910 comp rx=0 tx=0).stop 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 -- 172.21.15.71:0/908685932 shutdown_connections 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 --2- 172.21.15.71:0/908685932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e400c4960 0x7f0e400c72e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 --2- 172.21.15.71:0/908685932 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e4006fc60 0x7f0e400c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 --2- 172.21.15.71:0/908685932 >> v2:172.21.15.122:3300/0 conn(0x7f0e40151060 0x7f0e40154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 -- 172.21.15.71:0/908685932 >> 172.21.15.71:0/908685932 conn(0x7f0e400c93e0 msgr2=0x7f0e400c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 -- 172.21.15.71:0/908685932 shutdown_connections 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.685+0000 7f0e47495640 1 -- 172.21.15.71:0/908685932 wait complete. 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.686+0000 7f0e47495640 1 Processor -- start 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.686+0000 7f0e47495640 1 -- start start 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e47495640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e4006fc60 0x7f0e40157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e47495640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e400c4960 0x7f0e401eea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e47495640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e40151060 0x7f0e40154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e47495640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400c9090 con 0x7f0e40151060 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e47495640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400c5260 con 0x7f0e4006fc60 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e47495640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0e400c9210 con 0x7f0e400c4960 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45a32640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e40151060 0x7f0e40154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45a32640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e40151060 0x7f0e40154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45674/0 (socket says 172.21.15.71:45674) 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45a32640 1 -- 172.21.15.71:0/1909804722 learned_addr learned my addr 172.21.15.71:0/1909804722 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:20.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e44a30640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e400c4960 0x7f0e401eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45231640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e4006fc60 0x7f0e40157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45a32640 1 -- 172.21.15.71:0/1909804722 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e400c4960 msgr2=0x7f0e401eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45a32640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e400c4960 0x7f0e401eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45a32640 1 -- 172.21.15.71:0/1909804722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e4006fc60 msgr2=0x7f0e40157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45a32640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e4006fc60 0x7f0e40157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.687+0000 7f0e45a32640 1 -- 172.21.15.71:0/1909804722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e401ef250 con 0x7f0e40151060 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.688+0000 7f0e45a32640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e40151060 0x7f0e40154b10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e3802b950 tx=0x7f0e380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.688+0000 7f0e35ffb640 1 -- 172.21.15.71:0/1909804722 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38040070 con 0x7f0e40151060 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.688+0000 7f0e35ffb640 1 -- 172.21.15.71:0/1909804722 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3803a420 con 0x7f0e40151060 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.688+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e40157ad0 con 0x7f0e40151060 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.688+0000 7f0e35ffb640 1 -- 172.21.15.71:0/1909804722 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38035040 con 0x7f0e40151060 2024-04-16T15:40:20.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.688+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e40157db0 con 0x7f0e40151060 2024-04-16T15:40:20.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.689+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e40068cf0 con 0x7f0e40151060 2024-04-16T15:40:20.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.689+0000 7f0e35ffb640 1 -- 172.21.15.71:0/1909804722 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e38038020 con 0x7f0e40151060 2024-04-16T15:40:20.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.690+0000 7f0e35ffb640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0e2406c8f0 0x7f0e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.690+0000 7f0e35ffb640 1 -- 172.21.15.71:0/1909804722 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0e380ad650 con 0x7f0e40151060 2024-04-16T15:40:20.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.690+0000 7f0e45231640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0e2406c8f0 0x7f0e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.690+0000 7f0e45231640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0e2406c8f0 0x7f0e2406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0e400af1a0 tx=0x7f0e3c007710 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.694+0000 7f0e35ffb640 1 -- 172.21.15.71:0/1909804722 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e3807b810 con 0x7f0e40151060 2024-04-16T15:40:20.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.868+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e400c3ad0 con 0x7f0e40151060 2024-04-16T15:40:20.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.869+0000 7f0e35ffb640 1 -- 172.21.15.71:0/1909804722 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7f0e3807fb10 con 0x7f0e40151060 2024-04-16T15:40:20.868 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:20.868 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":16,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:13.860214+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:20.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0e2406c8f0 msgr2=0x7f0e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0e2406c8f0 0x7f0e2406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0e400af1a0 tx=0x7f0e3c007710 comp rx=0 tx=0).stop 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e40151060 msgr2=0x7f0e40154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e40151060 0x7f0e40154b10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e3802b950 tx=0x7f0e380046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 shutdown_connections 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0e2406c8f0 0x7f0e2406edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e40151060 0x7f0e40154b10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0e400c4960 0x7f0e401eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 --2- 172.21.15.71:0/1909804722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e4006fc60 0x7f0e40157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 >> 172.21.15.71:0/1909804722 conn(0x7f0e400c93e0 msgr2=0x7f0e400c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 shutdown_connections 2024-04-16T15:40:20.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.871+0000 7f0e47495640 1 -- 172.21.15.71:0/1909804722 wait complete. 2024-04-16T15:40:20.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T15:40:20.881 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:20.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.988+0000 7f0ec33f2640 1 Processor -- start 2024-04-16T15:40:20.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.988+0000 7f0ec33f2640 1 -- start start 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.989+0000 7f0ec33f2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0ebc153430 0x7f0ebc1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.989+0000 7f0ec33f2640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ebc156c70 con 0x7f0ebc156770 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.989+0000 7f0ec33f2640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ebc0c5ec0 con 0x7f0ebc157720 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.989+0000 7f0ec33f2640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f0ebc0c67f0 con 0x7f0ebc153430 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.989+0000 7f0ec198f640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f0ebc156770 0x7f0ebc160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:58616/0 (socket says 172.21.15.71:58616) 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.989+0000 7f0ec198f640 1 -- 172.21.15.71:0/1091765335 learned_addr learned my addr 172.21.15.71:0/1091765335 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 940697566 0 0) 0x7f0ebc156c70 con 0x7f0ebc156770 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec118e640 1 --2- 172.21.15.71:0/1091765335 >> v2:172.21.15.122:3300/0 conn(0x7f0ebc153430 0x7f0ebc1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea8003520 con 0x7f0ebc156770 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2879461795 0 0) 0x7f0ea8003520 con 0x7f0ebc156770 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 >> v2:172.21.15.122:3300/0 conn(0x7f0ebc153430 msgr2=0x7f0ebc1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:20.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 --2- 172.21.15.71:0/1091765335 >> v2:172.21.15.122:3300/0 conn(0x7f0ebc153430 0x7f0ebc1571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 >> v1:172.21.15.122:6789/0 conn(0x7f0ebc157720 legacy=0x7f0ebc0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc0c61a0 con 0x7f0ebc156770 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f0eb4001670 con 0x7f0ebc156770 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 >> v1:172.21.15.71:6789/0 conn(0x7f0ebc156770 legacy=0x7f0ebc160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ea80039b0 0x7f0ea8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ea8004800 0x7f0ea8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ea800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc0c67f0 con 0x7f0ea80051c0 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea800c160 con 0x7f0ea8004800 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea800c360 con 0x7f0ea80039b0 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec198f640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ea800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec098d640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ea8004800 0x7f0ea8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec118e640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ea80039b0 0x7f0ea8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec198f640 1 -- 172.21.15.71:0/1091765335 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ea80039b0 msgr2=0x7f0ea8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec198f640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ea80039b0 0x7f0ea8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec198f640 1 -- 172.21.15.71:0/1091765335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ea8004800 msgr2=0x7f0ea8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec198f640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ea8004800 0x7f0ea8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.990+0000 7f0ec198f640 1 -- 172.21.15.71:0/1091765335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea8003520 con 0x7f0ea80051c0 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.991+0000 7f0ec198f640 1 -- 172.21.15.71:0/1091765335 reap_dead start 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.991+0000 7f0ec198f640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ea800b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400c240 tx=0x7f0ea8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e372c5c77a89612 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.991+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4007d10 con 0x7f0ea80051c0 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.991+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb4030070 con 0x7f0ea80051c0 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.991+0000 7f0eb3fff640 1 -- 172.21.15.71:0/1091765335 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb402cd40 con 0x7f0ea80051c0 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.992+0000 7f0ec33f2640 1 -- 172.21.15.71:0/1091765335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 msgr2=0x7f0ea800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.992+0000 7f0ec33f2640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ea800b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400c240 tx=0x7f0ea8005830 comp rx=0 tx=0).stop 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.992+0000 7f0ec33f2640 1 -- 172.21.15.71:0/1091765335 shutdown_connections 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.992+0000 7f0ec33f2640 1 --2- 172.21.15.71:0/1091765335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ea800b9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.992+0000 7f0ec33f2640 1 -- 172.21.15.71:0/1091765335 >> 172.21.15.71:0/1091765335 conn(0x7f0ebc0c7f20 msgr2=0x7f0ebc0c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:20.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.992+0000 7f0ec33f2640 1 -- 172.21.15.71:0/1091765335 shutdown_connections 2024-04-16T15:40:20.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.992+0000 7f0ec33f2640 1 -- 172.21.15.71:0/1091765335 wait complete. 2024-04-16T15:40:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.993+0000 7f0ec33f2640 1 Processor -- start 2024-04-16T15:40:20.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.993+0000 7f0ec33f2640 1 -- start start 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec33f2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ebc1eb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec33f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ebc0bea00 0x7f0ebc159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec33f2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ebc14ce30 0x7f0ebc1ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec33f2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc153990 con 0x7f0ea80051c0 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec33f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc153810 con 0x7f0ebc0bea00 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec33f2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0ebc0cb120 con 0x7f0ebc14ce30 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ebc1eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ebc1eb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45692/0 (socket says 172.21.15.71:45692) 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 -- 172.21.15.71:0/864114875 learned_addr learned my addr 172.21.15.71:0/864114875 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec098d640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ebc0bea00 0x7f0ebc159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec198f640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ebc14ce30 0x7f0ebc1ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 -- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ebc14ce30 msgr2=0x7f0ebc1ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ebc14ce30 0x7f0ebc1ef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 -- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ebc0bea00 msgr2=0x7f0ebc159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:20.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ebc0bea00 0x7f0ebc159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 -- 172.21.15.71:0/864114875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc14c510 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec198f640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ebc14ce30 0x7f0ebc1ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec118e640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ebc1eb560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009f90 tx=0x7f0eb800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0eb1ffb640 1 -- 172.21.15.71:0/864114875 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800fea0 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0eb1ffb640 1 -- 172.21.15.71:0/864114875 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb8015630 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc14c7f0 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0eb1ffb640 1 -- 172.21.15.71:0/864114875 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8015a20 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.994+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc1eead0 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.995+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ebc073f50 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.996+0000 7f0eb1ffb640 1 -- 172.21.15.71:0/864114875 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0eb8015bc0 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.996+0000 7f0eb1ffb640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0ea006c920 0x7f0ea006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.996+0000 7f0ec098d640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0ea006c920 0x7f0ea006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.996+0000 7f0eb1ffb640 1 -- 172.21.15.71:0/864114875 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0eb808c8b0 con 0x7f0ea80051c0 2024-04-16T15:40:20.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:20.997+0000 7f0ec098d640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0ea006c920 0x7f0ea006ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0bee00 tx=0x7f0eac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:20.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.001+0000 7f0eb1ffb640 1 -- 172.21.15.71:0/864114875 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0eb8019030 con 0x7f0ea80051c0 2024-04-16T15:40:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.174+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ebc068be0 con 0x7f0ea80051c0 2024-04-16T15:40:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.174+0000 7f0eb1ffb640 1 -- 172.21.15.71:0/864114875 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f0eb8010040 con 0x7f0ea80051c0 2024-04-16T15:40:21.173 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0ea006c920 msgr2=0x7f0ea006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0ea006c920 0x7f0ea006ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0bee00 tx=0x7f0eac009040 comp rx=0 tx=0).stop 2024-04-16T15:40:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 msgr2=0x7f0ebc1eb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ebc1eb560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009f90 tx=0x7f0eb800f580 comp rx=0 tx=0).stop 2024-04-16T15:40:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 shutdown_connections 2024-04-16T15:40:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0ea006c920 0x7f0ea006ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ebc14ce30 0x7f0ebc1ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ebc0bea00 0x7f0ebc159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 --2- 172.21.15.71:0/864114875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ea80051c0 0x7f0ebc1eb560 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 >> 172.21.15.71:0/864114875 conn(0x7f0ebc0c7f20 msgr2=0x7f0ebc14ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 shutdown_connections 2024-04-16T15:40:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.176+0000 7f0ec33f2640 1 -- 172.21.15.71:0/864114875 wait complete. 2024-04-16T15:40:21.186 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-16T15:40:21.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.289+0000 7fa3edd15640 1 Processor -- start 2024-04-16T15:40:21.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.289+0000 7fa3edd15640 1 -- start start 2024-04-16T15:40:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3edd15640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa3e8153570 0x7fa3e80c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3edd15640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c4350 0x7fa3e80c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3edd15640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa3e80c62e0 0x7fa3e8159740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3edd15640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e80c41a0 con 0x7fa3e80c62e0 2024-04-16T15:40:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3edd15640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e80c82d0 con 0x7fa3e8153570 2024-04-16T15:40:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3edd15640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e8155d70 con 0x7fa3e80c4350 2024-04-16T15:40:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3e7fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa3e80c62e0 0x7fa3e8159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3e7fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa3e80c62e0 0x7fa3e8159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45702/0 (socket says 172.21.15.71:45702) 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3e7fff640 1 -- 172.21.15.71:0/1854684596 learned_addr learned my addr 172.21.15.71:0/1854684596 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3e6ffd640 1 --2- 172.21.15.71:0/1854684596 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c4350 0x7fa3e80c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.290+0000 7fa3e77fe640 1 --2- 172.21.15.71:0/1854684596 >> v2:172.21.15.122:3300/0 conn(0x7fa3e8153570 0x7fa3e80c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e7fff640 1 -- 172.21.15.71:0/1854684596 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c4350 msgr2=0x7fa3e80c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e7fff640 1 --2- 172.21.15.71:0/1854684596 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c4350 0x7fa3e80c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e7fff640 1 -- 172.21.15.71:0/1854684596 >> v2:172.21.15.122:3300/0 conn(0x7fa3e8153570 msgr2=0x7fa3e80c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e7fff640 1 --2- 172.21.15.71:0/1854684596 >> v2:172.21.15.122:3300/0 conn(0x7fa3e8153570 0x7fa3e80c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e7fff640 1 -- 172.21.15.71:0/1854684596 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e80c8d50 con 0x7fa3e80c62e0 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e6ffd640 1 --2- 172.21.15.71:0/1854684596 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c4350 0x7fa3e80c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e77fe640 1 --2- 172.21.15.71:0/1854684596 >> v2:172.21.15.122:3300/0 conn(0x7fa3e8153570 0x7fa3e80c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e7fff640 1 --2- 172.21.15.71:0/1854684596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e80c62e0 0x7fa3e8159740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80c3d20 tx=0x7fa3e002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149c37f1e5c958bf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e67fc640 1 -- 172.21.15.71:0/1854684596 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e002c950 con 0x7fa3e80c62e0 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e67fc640 1 -- 172.21.15.71:0/1854684596 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e002caf0 con 0x7fa3e80c62e0 2024-04-16T15:40:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.291+0000 7fa3e67fc640 1 -- 172.21.15.71:0/1854684596 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00343f0 con 0x7fa3e80c62e0 2024-04-16T15:40:21.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 -- 172.21.15.71:0/1854684596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e80c62e0 msgr2=0x7fa3e8159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 --2- 172.21.15.71:0/1854684596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e80c62e0 0x7fa3e8159740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80c3d20 tx=0x7fa3e002bdb0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 -- 172.21.15.71:0/1854684596 shutdown_connections 2024-04-16T15:40:21.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 --2- 172.21.15.71:0/1854684596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e80c62e0 0x7fa3e8159740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 --2- 172.21.15.71:0/1854684596 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c4350 0x7fa3e80c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 --2- 172.21.15.71:0/1854684596 >> v2:172.21.15.122:3300/0 conn(0x7fa3e8153570 0x7fa3e80c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 -- 172.21.15.71:0/1854684596 >> 172.21.15.71:0/1854684596 conn(0x7fa3e80c96e0 msgr2=0x7fa3e80c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 -- 172.21.15.71:0/1854684596 shutdown_connections 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.292+0000 7fa3edd15640 1 -- 172.21.15.71:0/1854684596 wait complete. 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.293+0000 7fa3edd15640 1 Processor -- start 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.293+0000 7fa3edd15640 1 -- start start 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3edd15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3e80c4350 0x7fa3e80c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3edd15640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c62e0 0x7fa3e81485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3edd15640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e8153570 0x7fa3e8148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3edd15640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e8159c80 con 0x7fa3e8153570 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3edd15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e80c8ff0 con 0x7fa3e80c4350 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3edd15640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e8159e00 con 0x7fa3e80c62e0 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e8153570 0x7fa3e8148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e8153570 0x7fa3e8148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45704/0 (socket says 172.21.15.71:45704) 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e7fff640 1 -- 172.21.15.71:0/948130745 learned_addr learned my addr 172.21.15.71:0/948130745 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:21.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e77fe640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3e80c4350 0x7fa3e80c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e6ffd640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c62e0 0x7fa3e81485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e7fff640 1 -- 172.21.15.71:0/948130745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c62e0 msgr2=0x7fa3e81485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e7fff640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c62e0 0x7fa3e81485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e7fff640 1 -- 172.21.15.71:0/948130745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3e80c4350 msgr2=0x7fa3e80c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e7fff640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3e80c4350 0x7fa3e80c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.294+0000 7fa3e7fff640 1 -- 172.21.15.71:0/948130745 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e8149070 con 0x7fa3e8153570 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.295+0000 7fa3e7fff640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e8153570 0x7fa3e8148b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0036040 tx=0x7fa3e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.295+0000 7fa3cffff640 1 -- 172.21.15.71:0/948130745 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0043020 con 0x7fa3e8153570 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.295+0000 7fa3cffff640 1 -- 172.21.15.71:0/948130745 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e003e450 con 0x7fa3e8153570 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.295+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e80abfd0 con 0x7fa3e8153570 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.295+0000 7fa3cffff640 1 -- 172.21.15.71:0/948130745 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e003e840 con 0x7fa3e8153570 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.295+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e80c30e0 con 0x7fa3e8153570 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.296+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e8068d20 con 0x7fa3e8153570 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.296+0000 7fa3cffff640 1 -- 172.21.15.71:0/948130745 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3e0046020 con 0x7fa3e8153570 2024-04-16T15:40:21.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.296+0000 7fa3cffff640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa3d406c8f0 0x7fa3d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.297+0000 7fa3cffff640 1 -- 172.21.15.71:0/948130745 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa3e00adef0 con 0x7fa3e8153570 2024-04-16T15:40:21.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.297+0000 7fa3e77fe640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa3d406c8f0 0x7fa3d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.297+0000 7fa3e77fe640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa3d406c8f0 0x7fa3d406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8006fd0 tx=0x7fa3d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:21.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.301+0000 7fa3cffff640 1 -- 172.21.15.71:0/948130745 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3e004d030 con 0x7fa3e8153570 2024-04-16T15:40:21.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.478+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa3e80c7c50 con 0x7fa3e8153570 2024-04-16T15:40:21.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.479+0000 7fa3cffff640 1 -- 172.21.15.71:0/948130745 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fa3e007c0b0 con 0x7fa3e8153570 2024-04-16T15:40:21.478 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:21.478 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-04-16T15:40:21.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:40:21.479 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-04-16T15:40:21.479 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-04-16T15:40:21.479 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:40:21.479 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-04-16T15:40:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.480+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa3d406c8f0 msgr2=0x7fa3d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.480+0000 7fa3edd15640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa3d406c8f0 0x7fa3d406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8006fd0 tx=0x7fa3d8008040 comp rx=0 tx=0).stop 2024-04-16T15:40:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.480+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e8153570 msgr2=0x7fa3e8148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.480+0000 7fa3edd15640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e8153570 0x7fa3e8148b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0036040 tx=0x7fa3e0004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.481+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 shutdown_connections 2024-04-16T15:40:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.481+0000 7fa3edd15640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa3d406c8f0 0x7fa3d406edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.481+0000 7fa3edd15640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e8153570 0x7fa3e8148b30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.481+0000 7fa3edd15640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3e80c62e0 0x7fa3e81485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.481+0000 7fa3edd15640 1 --2- 172.21.15.71:0/948130745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3e80c4350 0x7fa3e80c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.481+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 >> 172.21.15.71:0/948130745 conn(0x7fa3e80c96e0 msgr2=0x7fa3e80c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.481+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 shutdown_connections 2024-04-16T15:40:21.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.481+0000 7fa3edd15640 1 -- 172.21.15.71:0/948130745 wait complete. 2024-04-16T15:40:21.490 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-16T15:40:21.491 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-16T15:40:21.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.600+0000 7f1aa5bf4640 1 Processor -- start 2024-04-16T15:40:21.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.600+0000 7f1aa5bf4640 1 -- start start 2024-04-16T15:40:21.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1aa5bf4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1aa00a8640 0x7f1aa00bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1aa5bf4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0156740 0x7f1aa00c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1aa5bf4640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa00a8db0 con 0x7f1aa014fed0 2024-04-16T15:40:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1aa5bf4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa00bfd20 con 0x7f1aa00a8640 2024-04-16T15:40:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1aa5bf4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa00c0510 con 0x7f1aa0156740 2024-04-16T15:40:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9f7fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f1aa014fed0 0x7f1aa00c72a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:58622/0 (socket says 172.21.15.71:58622) 2024-04-16T15:40:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9f7fe640 1 -- 172.21.15.71:0/4188785784 learned_addr learned my addr 172.21.15.71:0/4188785784 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9ffff640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0156740 0x7f1aa00c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9effd640 1 --2- 172.21.15.71:0/4188785784 >> v2:172.21.15.122:3300/0 conn(0x7f1aa00a8640 0x7f1aa00bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055268530 0 0) 0x7f1a90001490 con 0x7f1aa014fed0 2024-04-16T15:40:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a88003520 con 0x7f1aa014fed0 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3773494299 0 0) 0x7f1a88003520 con 0x7f1aa014fed0 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0156740 msgr2=0x7f1aa00c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0156740 0x7f1aa00c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 >> v2:172.21.15.122:3300/0 conn(0x7f1aa00a8640 msgr2=0x7f1aa00bf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 --2- 172.21.15.71:0/4188785784 >> v2:172.21.15.122:3300/0 conn(0x7f1aa00a8640 0x7f1aa00bf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa00bff50 con 0x7f1aa014fed0 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f1a90001bc0 con 0x7f1aa014fed0 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 >> v1:172.21.15.71:6789/0 conn(0x7f1aa014fed0 legacy=0x7f1aa00c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0156740 0x7f1a880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88003f30 0x7f1a88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a88004d40 0x7f1a880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa00c0510 con 0x7f1a88004d40 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8800b9c0 con 0x7f1a88003f30 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.601+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8800bb40 con 0x7f1aa0156740 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.602+0000 7f1a9ffff640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a88004d40 0x7f1a880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.602+0000 7f1a9f7fe640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0156740 0x7f1a880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.602+0000 7f1a9effd640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88003f30 0x7f1a88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.602+0000 7f1a9ffff640 1 -- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0156740 msgr2=0x7f1a880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.602+0000 7f1a9ffff640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0156740 0x7f1a880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.602+0000 7f1a9ffff640 1 -- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88003f30 msgr2=0x7f1a88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.602+0000 7f1a9ffff640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88003f30 0x7f1a88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.602+0000 7f1a9ffff640 1 -- 172.21.15.71:0/4188785784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a88003520 con 0x7f1a88004d40 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1a9ffff640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a88004d40 0x7f1a880051c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009ac0 tx=0x7f1a88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a057b0ad38e9bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98039070 con 0x7f1a88004d40 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9802c6b0 con 0x7f1a88004d40 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1a9e7fc640 1 -- 172.21.15.71:0/4188785784 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98009690 con 0x7f1a88004d40 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/4188785784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a88004d40 msgr2=0x7f1a880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1aa5bf4640 1 --2- 172.21.15.71:0/4188785784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a88004d40 0x7f1a880051c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009ac0 tx=0x7f1a88005830 comp rx=0 tx=0).stop 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1a9ffff640 1 -- 172.21.15.71:0/4188785784 reap_dead start 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/4188785784 shutdown_connections 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/4188785784 >> 172.21.15.71:0/4188785784 conn(0x7f1aa00c5e20 msgr2=0x7f1aa00c6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/4188785784 shutdown_connections 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.603+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/4188785784 wait complete. 2024-04-16T15:40:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.604+0000 7f1aa5bf4640 1 Processor -- start 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1aa5bf4640 1 -- start start 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1aa5bf4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88004d40 0x7f1aa01ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1aa5bf4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa014b440 0x7f1aa01ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1aa5bf4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0154e60 0x7f1aa014a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1aa5bf4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0156220 con 0x7f1aa014b440 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1aa5bf4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa0155f40 con 0x7f1a88004d40 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1aa5bf4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa0156bd0 con 0x7f1aa0154e60 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa014b440 0x7f1aa01ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa014b440 0x7f1aa01ebcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45720/0 (socket says 172.21.15.71:45720) 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9effd640 1 -- 172.21.15.71:0/3465257606 learned_addr learned my addr 172.21.15.71:0/3465257606 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9ffff640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0154e60 0x7f1aa014a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9f7fe640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88004d40 0x7f1aa01ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9effd640 1 -- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0154e60 msgr2=0x7f1aa014a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9effd640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0154e60 0x7f1aa014a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9effd640 1 -- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88004d40 msgr2=0x7f1aa01ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.605+0000 7f1a9effd640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88004d40 0x7f1aa01ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.606+0000 7f1a9effd640 1 -- 172.21.15.71:0/3465257606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa01ee7d0 con 0x7f1aa014b440 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.606+0000 7f1a9f7fe640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88004d40 0x7f1aa01ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.606+0000 7f1a9effd640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa014b440 0x7f1aa01ebcc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a94011960 tx=0x7f1a94011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.606+0000 7f1a87fff640 1 -- 172.21.15.71:0/3465257606 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400faf0 con 0x7f1aa014b440 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.606+0000 7f1a87fff640 1 -- 172.21.15.71:0/3465257606 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9400fc90 con 0x7f1aa014b440 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.606+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa014b8d0 con 0x7f1aa014b440 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.606+0000 7f1a87fff640 1 -- 172.21.15.71:0/3465257606 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a94014440 con 0x7f1aa014b440 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.606+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa01eea90 con 0x7f1aa014b440 2024-04-16T15:40:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.607+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa0068be0 con 0x7f1aa014b440 2024-04-16T15:40:21.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.607+0000 7f1a87fff640 1 -- 172.21.15.71:0/3465257606 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a94012040 con 0x7f1aa014b440 2024-04-16T15:40:21.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.608+0000 7f1a87fff640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:21.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.608+0000 7f1a87fff640 1 -- 172.21.15.71:0/3465257606 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1a9408be50 con 0x7f1aa014b440 2024-04-16T15:40:21.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.608+0000 7f1a9f7fe640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:21.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.608+0000 7f1a9f7fe640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1a90001630 tx=0x7f1a90006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:21.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.612+0000 7f1a87fff640 1 -- 172.21.15.71:0/3465257606 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a94019030 con 0x7f1aa014b440 2024-04-16T15:40:21.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:21.790+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1aa00c8420 con 0x7f1aa014b440 2024-04-16T15:40:22.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.511+0000 7f1a87fff640 1 -- 172.21.15.71:0/3465257606 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f1a9405a010 con 0x7f1aa014b440 2024-04-16T15:40:22.510 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-04-16T15:40:22.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1a8c06c8f0 msgr2=0x7f1a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:22.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1a90001630 tx=0x7f1a90006040 comp rx=0 tx=0).stop 2024-04-16T15:40:22.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa014b440 msgr2=0x7f1aa01ebcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:22.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa014b440 0x7f1aa01ebcc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a94011960 tx=0x7f1a94011e20 comp rx=0 tx=0).stop 2024-04-16T15:40:22.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 shutdown_connections 2024-04-16T15:40:22.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1a8c06c8f0 0x7f1a8c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1aa0154e60 0x7f1aa014a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1aa014b440 0x7f1aa01ebcc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 --2- 172.21.15.71:0/3465257606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a88004d40 0x7f1aa01ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 >> 172.21.15.71:0/3465257606 conn(0x7f1aa00c5e20 msgr2=0x7f1aa0071660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:22.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.514+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 shutdown_connections 2024-04-16T15:40:22.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.515+0000 7f1aa5bf4640 1 -- 172.21.15.71:0/3465257606 wait complete. 2024-04-16T15:40:22.529 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-16T15:40:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.640+0000 7fa85f40d640 1 Processor -- start 2024-04-16T15:40:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.640+0000 7fa85f40d640 1 -- start start 2024-04-16T15:40:22.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa85f40d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa8580c94c0 0x7fa8581564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:22.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa85f40d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa85806e820 con 0x7fa8580c99d0 2024-04-16T15:40:22.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa85f40d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa858159fb0 con 0x7fa858153450 2024-04-16T15:40:22.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa85f40d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fa8580c8150 con 0x7fa8580c94c0 2024-04-16T15:40:22.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa85c9a8640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fa8580c99d0 0x7fa858155db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:58636/0 (socket says 172.21.15.71:58636) 2024-04-16T15:40:22.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa85c9a8640 1 -- 172.21.15.71:0/293365564 learned_addr learned my addr 172.21.15.71:0/293365564 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3608505629 0 0) 0x7fa85806e820 con 0x7fa8580c99d0 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa85d9aa640 1 --2- 172.21.15.71:0/293365564 >> v2:172.21.15.122:3300/0 conn(0x7fa8580c94c0 0x7fa8581564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa844003520 con 0x7fa8580c99d0 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 117229659 0 0) 0x7fa844003520 con 0x7fa8580c99d0 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 >> v2:172.21.15.122:3300/0 conn(0x7fa8580c94c0 msgr2=0x7fa8581564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 --2- 172.21.15.71:0/293365564 >> v2:172.21.15.122:3300/0 conn(0x7fa8580c94c0 0x7fa8581564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 >> v1:172.21.15.122:6789/0 conn(0x7fa858153450 legacy=0x7fa8580c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8580c8430 con 0x7fa8580c99d0 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fa848001670 con 0x7fa8580c99d0 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 >> v1:172.21.15.71:6789/0 conn(0x7fa8580c99d0 legacy=0x7fa858155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8440039b0 0x7fa844003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa844004800 0x7fa844004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8440053c0 0x7fa84400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8580c8150 con 0x7fa8440053c0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa84400c250 con 0x7fa844004800 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa84400c450 con 0x7fa8440039b0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.641+0000 7fa85d9aa640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8440053c0 0x7fa84400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85d1a9640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8440039b0 0x7fa844003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85c9a8640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa844004800 0x7fa844004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85d9aa640 1 -- 172.21.15.71:0/293365564 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8440039b0 msgr2=0x7fa844003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85d9aa640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8440039b0 0x7fa844003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85d9aa640 1 -- 172.21.15.71:0/293365564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa844004800 msgr2=0x7fa844004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85d9aa640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa844004800 0x7fa844004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85d9aa640 1 -- 172.21.15.71:0/293365564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa844003520 con 0x7fa8440053c0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85d9aa640 1 -- 172.21.15.71:0/293365564 reap_dead start 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.642+0000 7fa85d9aa640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8440053c0 0x7fa84400b9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa85000ba30 tx=0x7fa844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b3a79c68b641f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850036030 con 0x7fa8440053c0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa850004020 con 0x7fa8440053c0 2024-04-16T15:40:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa84ffff640 1 -- 172.21.15.71:0/293365564 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85000b600 con 0x7fa8440053c0 2024-04-16T15:40:22.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa85f40d640 1 -- 172.21.15.71:0/293365564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8440053c0 msgr2=0x7fa84400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:22.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa85f40d640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8440053c0 0x7fa84400b9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa85000ba30 tx=0x7fa844005830 comp rx=0 tx=0).stop 2024-04-16T15:40:22.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa85f40d640 1 -- 172.21.15.71:0/293365564 shutdown_connections 2024-04-16T15:40:22.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa85f40d640 1 --2- 172.21.15.71:0/293365564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8440053c0 0x7fa84400b9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa85f40d640 1 -- 172.21.15.71:0/293365564 >> 172.21.15.71:0/293365564 conn(0x7fa8580ad2f0 msgr2=0x7fa8580ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:22.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa85f40d640 1 -- 172.21.15.71:0/293365564 shutdown_connections 2024-04-16T15:40:22.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.643+0000 7fa85f40d640 1 -- 172.21.15.71:0/293365564 wait complete. 2024-04-16T15:40:22.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.644+0000 7fa85f40d640 1 Processor -- start 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.644+0000 7fa85f40d640 1 -- start start 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85f40d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8440053c0 0x7fa8580c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85f40d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8580c1280 0x7fa8581eef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85f40d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8581ef4c0 0x7fa85814c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85f40d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8580c60c0 con 0x7fa8580c1280 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85f40d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8580c5f40 con 0x7fa8440053c0 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85f40d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa8580c6240 con 0x7fa8581ef4c0 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85c9a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8580c1280 0x7fa8581eef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85c9a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8580c1280 0x7fa8581eef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45740/0 (socket says 172.21.15.71:45740) 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85c9a8640 1 -- 172.21.15.71:0/2615442639 learned_addr learned my addr 172.21.15.71:0/2615442639 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85d9aa640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8581ef4c0 0x7fa85814c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85c9a8640 1 -- 172.21.15.71:0/2615442639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8581ef4c0 msgr2=0x7fa85814c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85c9a8640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8581ef4c0 0x7fa85814c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85c9a8640 1 -- 172.21.15.71:0/2615442639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8440053c0 msgr2=0x7fa8580c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85c9a8640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8440053c0 0x7fa8580c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.645+0000 7fa85c9a8640 1 -- 172.21.15.71:0/2615442639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581efa00 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.646+0000 7fa85c9a8640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8580c1280 0x7fa8581eef80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa848004f20 tx=0x7fa8480074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.646+0000 7fa84dffb640 1 -- 172.21.15.71:0/2615442639 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84800f6d0 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.646+0000 7fa84dffb640 1 -- 172.21.15.71:0/2615442639 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa848018070 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.646+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85814dd10 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.646+0000 7fa84dffb640 1 -- 172.21.15.71:0/2615442639 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84800fad0 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.646+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85814e2d0 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.647+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa858074060 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.647+0000 7fa84dffb640 1 -- 172.21.15.71:0/2615442639 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa84800fc70 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.648+0000 7fa84dffb640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.648+0000 7fa85d1a9640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.648+0000 7fa84dffb640 1 -- 172.21.15.71:0/2615442639 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fa84808b8d0 con 0x7fa8580c1280 2024-04-16T15:40:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.648+0000 7fa85d1a9640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa83c06c8f0 0x7fa83c06edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa854001dd0 tx=0x7fa854009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.652+0000 7fa84dffb640 1 -- 172.21.15.71:0/2615442639 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa848059800 con 0x7fa8580c1280 2024-04-16T15:40:22.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:22.829+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa85814db20 con 0x7fa8580c1280 2024-04-16T15:40:23.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:40:23.392 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:40:23.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.513+0000 7fa84dffb640 1 -- 172.21.15.71:0/2615442639 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7fa84805db00 con 0x7fa8580c1280 2024-04-16T15:40:23.512 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-04-16T15:40:23.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.517+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa83c06c8f0 msgr2=0x7fa83c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:23.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.517+0000 7fa85f40d640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa83c06c8f0 0x7fa83c06edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa854001dd0 tx=0x7fa854009040 comp rx=0 tx=0).stop 2024-04-16T15:40:23.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.517+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8580c1280 msgr2=0x7fa8581eef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:23.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.517+0000 7fa85f40d640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8580c1280 0x7fa8581eef80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa848004f20 tx=0x7fa8480074f0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.518+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 shutdown_connections 2024-04-16T15:40:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.518+0000 7fa85f40d640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.518+0000 7fa85f40d640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8581ef4c0 0x7fa85814c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.518+0000 7fa85f40d640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8580c1280 0x7fa8581eef80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.518+0000 7fa85f40d640 1 --2- 172.21.15.71:0/2615442639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8440053c0 0x7fa8580c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.518+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 >> 172.21.15.71:0/2615442639 conn(0x7fa8580ad2f0 msgr2=0x7fa8580c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.518+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 shutdown_connections 2024-04-16T15:40:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.518+0000 7fa85f40d640 1 -- 172.21.15.71:0/2615442639 wait complete. 2024-04-16T15:40:23.539 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-16T15:40:23.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.646+0000 7fb9be33d640 1 Processor -- start 2024-04-16T15:40:23.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.646+0000 7fb9be33d640 1 -- start start 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9be33d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb9b80735c0 0x7fb9b80c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9be33d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b8155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9be33d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb9b8152c90 0x7fb9b8153120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9be33d640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b81595b0 con 0x7fb9b8152c90 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9be33d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b80c1340 con 0x7fb9b80735c0 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9be33d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b80c1620 con 0x7fb9b8159a40 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9bc8da640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb9b8152c90 0x7fb9b8153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9bc8da640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb9b8152c90 0x7fb9b8153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45742/0 (socket says 172.21.15.71:45742) 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9bc8da640 1 -- 172.21.15.71:0/708142892 learned_addr learned my addr 172.21.15.71:0/708142892 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9b7fff640 1 --2- 172.21.15.71:0/708142892 >> v2:172.21.15.122:3300/0 conn(0x7fb9b80735c0 0x7fb9b80c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:23.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9b77fe640 1 --2- 172.21.15.71:0/708142892 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b8155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9bc8da640 1 -- 172.21.15.71:0/708142892 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 msgr2=0x7fb9b8155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9bc8da640 1 --2- 172.21.15.71:0/708142892 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b8155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9bc8da640 1 -- 172.21.15.71:0/708142892 >> v2:172.21.15.122:3300/0 conn(0x7fb9b80735c0 msgr2=0x7fb9b80c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9bc8da640 1 --2- 172.21.15.71:0/708142892 >> v2:172.21.15.122:3300/0 conn(0x7fb9b80735c0 0x7fb9b80c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9bc8da640 1 -- 172.21.15.71:0/708142892 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b8159240 con 0x7fb9b8152c90 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.647+0000 7fb9b7fff640 1 --2- 172.21.15.71:0/708142892 >> v2:172.21.15.122:3300/0 conn(0x7fb9b80735c0 0x7fb9b80c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.648+0000 7fb9bc8da640 1 --2- 172.21.15.71:0/708142892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b8153120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80c7a70 tx=0x7fb9b002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ee8cd6bd0c573 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.648+0000 7fb9b6ffd640 1 -- 172.21.15.71:0/708142892 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b002c950 con 0x7fb9b8152c90 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.648+0000 7fb9b6ffd640 1 -- 172.21.15.71:0/708142892 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b002caf0 con 0x7fb9b8152c90 2024-04-16T15:40:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.648+0000 7fb9b6ffd640 1 -- 172.21.15.71:0/708142892 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00343f0 con 0x7fb9b8152c90 2024-04-16T15:40:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 -- 172.21.15.71:0/708142892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 msgr2=0x7fb9b8153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 --2- 172.21.15.71:0/708142892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b8153120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80c7a70 tx=0x7fb9b002bdb0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 -- 172.21.15.71:0/708142892 shutdown_connections 2024-04-16T15:40:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 --2- 172.21.15.71:0/708142892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b8153120 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 --2- 172.21.15.71:0/708142892 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b8155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 --2- 172.21.15.71:0/708142892 >> v2:172.21.15.122:3300/0 conn(0x7fb9b80735c0 0x7fb9b80c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 -- 172.21.15.71:0/708142892 >> 172.21.15.71:0/708142892 conn(0x7fb9b806b930 msgr2=0x7fb9b806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 -- 172.21.15.71:0/708142892 shutdown_connections 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.649+0000 7fb9be33d640 1 -- 172.21.15.71:0/708142892 wait complete. 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.650+0000 7fb9be33d640 1 Processor -- start 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.650+0000 7fb9be33d640 1 -- start start 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9be33d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9b80735c0 0x7fb9b8151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9be33d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b81efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9be33d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b81f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9be33d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b815a020 con 0x7fb9b8152c90 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9be33d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8159ea0 con 0x7fb9b80735c0 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9be33d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b80a7480 con 0x7fb9b8159a40 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b81efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b81efe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45748/0 (socket says 172.21.15.71:45748) 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 -- 172.21.15.71:0/2677726205 learned_addr learned my addr 172.21.15.71:0/2677726205 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b7fff640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9b80735c0 0x7fb9b8151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9bc8da640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b81f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 -- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 msgr2=0x7fb9b81f03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b81f03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 -- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9b80735c0 msgr2=0x7fb9b8151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9b80735c0 0x7fb9b8151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 -- 172.21.15.71:0/2677726205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b81541f0 con 0x7fb9b8152c90 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b7fff640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9b80735c0 0x7fb9b8151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9bc8da640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b81f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b77fe640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b81efe60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00c7f0 tx=0x7fb9ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b4ff9640 1 -- 172.21.15.71:0/2677726205 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac015070 con 0x7fb9b8152c90 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b4ff9640 1 -- 172.21.15.71:0/2677726205 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac004240 con 0x7fb9b8152c90 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9b4ff9640 1 -- 172.21.15.71:0/2677726205 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac007620 con 0x7fb9b8152c90 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.651+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b81544d0 con 0x7fb9b8152c90 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.652+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b80ab720 con 0x7fb9b8152c90 2024-04-16T15:40:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.652+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b81efc00 con 0x7fb9b8152c90 2024-04-16T15:40:23.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.653+0000 7fb9b4ff9640 1 -- 172.21.15.71:0/2677726205 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9ac0079d0 con 0x7fb9b8152c90 2024-04-16T15:40:23.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.653+0000 7fb9b4ff9640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9a406c920 0x7fb9a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:23.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.654+0000 7fb9b4ff9640 1 -- 172.21.15.71:0/2677726205 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fb9ac08be40 con 0x7fb9b8152c90 2024-04-16T15:40:23.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.654+0000 7fb9b7fff640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9a406c920 0x7fb9a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:23.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.654+0000 7fb9b7fff640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9a406c920 0x7fb9a406ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800a990 tx=0x7fb9a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.658+0000 7fb9b4ff9640 1 -- 172.21.15.71:0/2677726205 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9ac05a440 con 0x7fb9b8152c90 2024-04-16T15:40:23.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:23.876+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb9b806d240 con 0x7fb9b8152c90 2024-04-16T15:40:24.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.520+0000 7fb9b4ff9640 1 -- 172.21.15.71:0/2677726205 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fb9b806d240 con 0x7fb9b8152c90 2024-04-16T15:40:24.519 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 2 and data pool 3 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9a406c920 msgr2=0x7fb9a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9a406c920 0x7fb9a406ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800a990 tx=0x7fb9a8008040 comp rx=0 tx=0).stop 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 msgr2=0x7fb9b81efe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b81efe60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00c7f0 tx=0x7fb9ac00ccb0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 shutdown_connections 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9a406c920 0x7fb9a406ede0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb9b8159a40 0x7fb9b81f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b8152c90 0x7fb9b81efe60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 --2- 172.21.15.71:0/2677726205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9b80735c0 0x7fb9b8151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.524+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 >> 172.21.15.71:0/2677726205 conn(0x7fb9b806b930 msgr2=0x7fb9b80c60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:24.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.525+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 shutdown_connections 2024-04-16T15:40:24.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.525+0000 7fb9be33d640 1 -- 172.21.15.71:0/2677726205 wait complete. 2024-04-16T15:40:24.551 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-16T15:40:24.552 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-16T15:40:24.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.662+0000 7fe1ed89a640 1 Processor -- start 2024-04-16T15:40:24.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.662+0000 7fe1ed89a640 1 -- start start 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1ed89a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe1e8159380 0x7fe1e8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1ed89a640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe1e80c3080 0x7fe1e806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1ed89a640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e8159cc0 con 0x7fe1e8159760 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1ed89a640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e80740b0 con 0x7fe1e80c3080 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1ed89a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e8159fa0 con 0x7fe1e8159380 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e67fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe1e80c3080 0x7fe1e806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e77fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fe1e8159760 0x7fe1e80739a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:58642/0 (socket says 172.21.15.71:58642) 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e67fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe1e80c3080 0x7fe1e806f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45750/0 (socket says 172.21.15.71:45750) 2024-04-16T15:40:24.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e77fe640 1 -- 172.21.15.71:0/217756608 learned_addr learned my addr 172.21.15.71:0/217756608 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e6ffd640 1 --2- 172.21.15.71:0/217756608 >> v2:172.21.15.122:3300/0 conn(0x7fe1e8159380 0x7fe1e8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e67fc640 1 -- 172.21.15.71:0/217756608 >> v2:172.21.15.122:3300/0 conn(0x7fe1e8159380 msgr2=0x7fe1e8072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e67fc640 1 --2- 172.21.15.71:0/217756608 >> v2:172.21.15.122:3300/0 conn(0x7fe1e8159380 0x7fe1e8072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e67fc640 1 -- 172.21.15.71:0/217756608 >> v1:172.21.15.71:6789/0 conn(0x7fe1e8159760 legacy=0x7fe1e80739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e5ffb640 1 -- 172.21.15.71:0/217756608 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789561601 0 0) 0x7fe1e8159cc0 con 0x7fe1e8159760 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.663+0000 7fe1e67fc640 1 -- 172.21.15.71:0/217756608 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e80c2350 con 0x7fe1e80c3080 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.664+0000 7fe1e67fc640 1 --2- 172.21.15.71:0/217756608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e806f9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80746b0 tx=0x7fe1dc02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa99b82079aa9714 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.664+0000 7fe1e5ffb640 1 -- 172.21.15.71:0/217756608 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc038070 con 0x7fe1e80c3080 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.664+0000 7fe1e5ffb640 1 -- 172.21.15.71:0/217756608 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1dc031710 con 0x7fe1e80c3080 2024-04-16T15:40:24.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.664+0000 7fe1e5ffb640 1 -- 172.21.15.71:0/217756608 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc031ae0 con 0x7fe1e80c3080 2024-04-16T15:40:24.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.665+0000 7fe1ed89a640 1 -- 172.21.15.71:0/217756608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 msgr2=0x7fe1e806f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:24.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.665+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/217756608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e806f9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80746b0 tx=0x7fe1dc02b7b0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.665+0000 7fe1ed89a640 1 -- 172.21.15.71:0/217756608 shutdown_connections 2024-04-16T15:40:24.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.665+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/217756608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e806f9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.665+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/217756608 >> v2:172.21.15.122:3300/0 conn(0x7fe1e8159380 0x7fe1e8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.665+0000 7fe1ed89a640 1 -- 172.21.15.71:0/217756608 >> 172.21.15.71:0/217756608 conn(0x7fe1e80aa220 msgr2=0x7fe1e80aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.665+0000 7fe1ed89a640 1 -- 172.21.15.71:0/217756608 shutdown_connections 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.665+0000 7fe1ed89a640 1 -- 172.21.15.71:0/217756608 wait complete. 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.666+0000 7fe1ed89a640 1 Processor -- start 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.666+0000 7fe1ed89a640 1 -- start start 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1ed89a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e80c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1ed89a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1e8159380 0x7fe1e806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1ed89a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1e8159760 0x7fe1e806d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1ed89a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e8152fd0 con 0x7fe1e80c3080 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1ed89a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e8152e50 con 0x7fe1e8159380 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1ed89a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e8153150 con 0x7fe1e8159760 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e80c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e80c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45762/0 (socket says 172.21.15.71:45762) 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e6ffd640 1 -- 172.21.15.71:0/3085347543 learned_addr learned my addr 172.21.15.71:0/3085347543 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e67fc640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1e8159380 0x7fe1e806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:24.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e77fe640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1e8159760 0x7fe1e806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e6ffd640 1 -- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1e8159760 msgr2=0x7fe1e806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e6ffd640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1e8159760 0x7fe1e806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e6ffd640 1 -- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1e8159380 msgr2=0x7fe1e806d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e6ffd640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1e8159380 0x7fe1e806d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e6ffd640 1 -- 172.21.15.71:0/3085347543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e81ee940 con 0x7fe1e80c3080 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.667+0000 7fe1e77fe640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1e8159760 0x7fe1e806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.668+0000 7fe1e6ffd640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e80c7840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800d800 tx=0x7fe1d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.668+0000 7fe1cffff640 1 -- 172.21.15.71:0/3085347543 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800b6c0 con 0x7fe1e80c3080 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.668+0000 7fe1cffff640 1 -- 172.21.15.71:0/3085347543 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d8019070 con 0x7fe1e80c3080 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.668+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e81eec20 con 0x7fe1e80c3080 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.668+0000 7fe1cffff640 1 -- 172.21.15.71:0/3085347543 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800ba80 con 0x7fe1e80c3080 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.668+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e80c7d80 con 0x7fe1e80c3080 2024-04-16T15:40:24.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.669+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e8068be0 con 0x7fe1e80c3080 2024-04-16T15:40:24.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.670+0000 7fe1cffff640 1 -- 172.21.15.71:0/3085347543 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1d800bc20 con 0x7fe1e80c3080 2024-04-16T15:40:24.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.670+0000 7fe1cffff640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe1d406c8f0 0x7fe1d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:24.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.670+0000 7fe1cffff640 1 -- 172.21.15.71:0/3085347543 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fe1d808bee0 con 0x7fe1e80c3080 2024-04-16T15:40:24.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.670+0000 7fe1e67fc640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe1d406c8f0 0x7fe1d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:24.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.671+0000 7fe1e67fc640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe1d406c8f0 0x7fe1d406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc035000 tx=0x7fe1dc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:24.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.675+0000 7fe1cffff640 1 -- 172.21.15.71:0/3085347543 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1d805a3a0 con 0x7fe1e80c3080 2024-04-16T15:40:24.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:24.865+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe1e814f9b0 con 0x7fe1e80c3080 2024-04-16T15:40:25.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.537+0000 7fe1cffff640 1 -- 172.21.15.71:0/3085347543 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7fe1d8059ca0 con 0x7fe1e80c3080 2024-04-16T15:40:25.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe1d406c8f0 msgr2=0x7fe1d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:25.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe1d406c8f0 0x7fe1d406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc035000 tx=0x7fe1dc008030 comp rx=0 tx=0).stop 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 msgr2=0x7fe1e80c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e80c7840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800d800 tx=0x7fe1d800dcc0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 shutdown_connections 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe1d406c8f0 0x7fe1d406edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe1e8159760 0x7fe1e806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1e8159380 0x7fe1e806d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 --2- 172.21.15.71:0/3085347543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1e80c3080 0x7fe1e80c7840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.540+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 >> 172.21.15.71:0/3085347543 conn(0x7fe1e80aa220 msgr2=0x7fe1e81521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.541+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 shutdown_connections 2024-04-16T15:40:25.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.541+0000 7fe1ed89a640 1 -- 172.21.15.71:0/3085347543 wait complete. 2024-04-16T15:40:25.552 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-16T15:40:25.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.666+0000 7fa6c862a640 1 Processor -- start 2024-04-16T15:40:25.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.666+0000 7fa6c862a640 1 -- start start 2024-04-16T15:40:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c862a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa6c006ea70 0x7fa6c0156d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c862a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 0x7fa6c00aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c862a640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa6c00aa970 0x7fa6c014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c862a640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c006ff90 con 0x7fa6c00aa970 2024-04-16T15:40:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c862a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c006fcb0 con 0x7fa6c006ea70 2024-04-16T15:40:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c862a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c0156280 con 0x7fa6c00c2790 2024-04-16T15:40:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c6bc7640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa6c00aa970 0x7fa6c014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c6bc7640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa6c00aa970 0x7fa6c014fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45772/0 (socket says 172.21.15.71:45772) 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.667+0000 7fa6c6bc7640 1 -- 172.21.15.71:0/3553099692 learned_addr learned my addr 172.21.15.71:0/3553099692 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.668+0000 7fa6c63c6640 1 --2- 172.21.15.71:0/3553099692 >> v2:172.21.15.122:3300/0 conn(0x7fa6c006ea70 0x7fa6c0156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.668+0000 7fa6c6bc7640 1 -- 172.21.15.71:0/3553099692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 msgr2=0x7fa6c00aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.668+0000 7fa6c6bc7640 1 --2- 172.21.15.71:0/3553099692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 0x7fa6c00aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.668+0000 7fa6c6bc7640 1 -- 172.21.15.71:0/3553099692 >> v2:172.21.15.122:3300/0 conn(0x7fa6c006ea70 msgr2=0x7fa6c0156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.668+0000 7fa6c6bc7640 1 --2- 172.21.15.71:0/3553099692 >> v2:172.21.15.122:3300/0 conn(0x7fa6c006ea70 0x7fa6c0156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.668+0000 7fa6c6bc7640 1 -- 172.21.15.71:0/3553099692 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c0156560 con 0x7fa6c00aa970 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.668+0000 7fa6c63c6640 1 --2- 172.21.15.71:0/3553099692 >> v2:172.21.15.122:3300/0 conn(0x7fa6c006ea70 0x7fa6c0156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.668+0000 7fa6c6bc7640 1 --2- 172.21.15.71:0/3553099692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c014fe70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6c006f930 tx=0x7fa6b802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cef6fdfe3786345 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c53c4640 1 -- 172.21.15.71:0/3553099692 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8038070 con 0x7fa6c00aa970 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c53c4640 1 -- 172.21.15.71:0/3553099692 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b802f050 con 0x7fa6c00aa970 2024-04-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c53c4640 1 -- 172.21.15.71:0/3553099692 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8031970 con 0x7fa6c00aa970 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 -- 172.21.15.71:0/3553099692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 msgr2=0x7fa6c014fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 --2- 172.21.15.71:0/3553099692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c014fe70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6c006f930 tx=0x7fa6b802b7f0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 -- 172.21.15.71:0/3553099692 shutdown_connections 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 --2- 172.21.15.71:0/3553099692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c014fe70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 --2- 172.21.15.71:0/3553099692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 0x7fa6c00aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 --2- 172.21.15.71:0/3553099692 >> v2:172.21.15.122:3300/0 conn(0x7fa6c006ea70 0x7fa6c0156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 -- 172.21.15.71:0/3553099692 >> 172.21.15.71:0/3553099692 conn(0x7fa6c00a9e20 msgr2=0x7fa6c00c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 -- 172.21.15.71:0/3553099692 shutdown_connections 2024-04-16T15:40:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.669+0000 7fa6c862a640 1 -- 172.21.15.71:0/3553099692 wait complete. 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.670+0000 7fa6c862a640 1 Processor -- start 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.670+0000 7fa6c862a640 1 -- start start 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c862a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c006ea70 0x7fa6c01dee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c862a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c01e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c862a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 0x7fa6c01ea040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c862a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c00c2bf0 con 0x7fa6c00aa970 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c862a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c006f980 con 0x7fa6c006ea70 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c862a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c00c3a20 con 0x7fa6c00c2790 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c5bc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c01e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c5bc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c01e9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45786/0 (socket says 172.21.15.71:45786) 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c5bc5640 1 -- 172.21.15.71:0/2690957599 learned_addr learned my addr 172.21.15.71:0/2690957599 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c63c6640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c006ea70 0x7fa6c01dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.671+0000 7fa6c6bc7640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 0x7fa6c01ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6c5bc5640 1 -- 172.21.15.71:0/2690957599 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 msgr2=0x7fa6c01ea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6c5bc5640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 0x7fa6c01ea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6c5bc5640 1 -- 172.21.15.71:0/2690957599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c006ea70 msgr2=0x7fa6c01dee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6c5bc5640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c006ea70 0x7fa6c01dee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6c5bc5640 1 -- 172.21.15.71:0/2690957599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01ed7c0 con 0x7fa6c00aa970 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6c5bc5640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c01e9b00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b410 tx=0x7fa6b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6beffd640 1 -- 172.21.15.71:0/2690957599 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4015020 con 0x7fa6c00aa970 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6beffd640 1 -- 172.21.15.71:0/2690957599 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b4011850 con 0x7fa6c00aa970 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c00bc5b0 con 0x7fa6c00aa970 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6beffd640 1 -- 172.21.15.71:0/2690957599 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4011c40 con 0x7fa6c00aa970 2024-04-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.672+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c00bc890 con 0x7fa6c00aa970 2024-04-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.673+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c00bc3a0 con 0x7fa6c00aa970 2024-04-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.673+0000 7fa6beffd640 1 -- 172.21.15.71:0/2690957599 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6b4018020 con 0x7fa6c00aa970 2024-04-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.674+0000 7fa6beffd640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa6ac06c8f0 0x7fa6ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.674+0000 7fa6c63c6640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa6ac06c8f0 0x7fa6ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.674+0000 7fa6beffd640 1 -- 172.21.15.71:0/2690957599 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7fa6b408ba10 con 0x7fa6c00aa970 2024-04-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.674+0000 7fa6c63c6640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa6ac06c8f0 0x7fa6ac06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0155c60 tx=0x7fa6b0007770 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:25.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.678+0000 7fa6beffd640 1 -- 172.21.15.71:0/2690957599 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6b401f020 con 0x7fa6c00aa970 2024-04-16T15:40:25.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:25.854+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa6c0068cf0 con 0x7fa6c00aa970 2024-04-16T15:40:27.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.540+0000 7fa6beffd640 1 -- 172.21.15.71:0/2690957599 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7fa6b4059750 con 0x7fa6c00aa970 2024-04-16T15:40:27.539 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data_ec' created 2024-04-16T15:40:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa6ac06c8f0 msgr2=0x7fa6ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa6ac06c8f0 0x7fa6ac06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0155c60 tx=0x7fa6b0007770 comp rx=0 tx=0).stop 2024-04-16T15:40:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 msgr2=0x7fa6c01e9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c01e9b00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b410 tx=0x7fa6b400b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 shutdown_connections 2024-04-16T15:40:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa6ac06c8f0 0x7fa6ac06edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa6c00c2790 0x7fa6c01ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6c00aa970 0x7fa6c01e9b00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 --2- 172.21.15.71:0/2690957599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6c006ea70 0x7fa6c01dee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 >> 172.21.15.71:0/2690957599 conn(0x7fa6c00a9e20 msgr2=0x7fa6c0154360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.544+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 shutdown_connections 2024-04-16T15:40:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.545+0000 7fa6c862a640 1 -- 172.21.15.71:0/2690957599 wait complete. 2024-04-16T15:40:27.563 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-16T15:40:27.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.675+0000 7f51a8f82640 1 Processor -- start 2024-04-16T15:40:27.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.675+0000 7f51a8f82640 1 -- start start 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a8f82640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f51a40a8640 0x7f51a40bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a8f82640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f51a4156740 0x7f51a40c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a8f82640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a40a8db0 con 0x7f51a414fed0 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a8f82640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f51a40bfd20 con 0x7f51a4156740 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a8f82640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f51a40c0510 con 0x7f51a40a8640 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a2d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f51a4156740 0x7f51a40c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a259c640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f51a414fed0 0x7f51a40c72a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56018/0 (socket says 172.21.15.71:56018) 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a259c640 1 -- 172.21.15.71:0/1975207433 learned_addr learned my addr 172.21.15.71:0/1975207433 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a2d9d640 1 -- 172.21.15.71:0/1975207433 >> v2:172.21.15.122:3300/0 conn(0x7f51a40a8640 msgr2=0x7f51a40bf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a1d9b640 1 --2- 172.21.15.71:0/1975207433 >> v2:172.21.15.122:3300/0 conn(0x7f51a40a8640 0x7f51a40bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a159a640 1 -- 172.21.15.71:0/1975207433 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3746383919 0 0) 0x7f5194001490 con 0x7f51a414fed0 2024-04-16T15:40:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a2d9d640 1 --2- 172.21.15.71:0/1975207433 >> v2:172.21.15.122:3300/0 conn(0x7f51a40a8640 0x7f51a40bf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a2d9d640 1 -- 172.21.15.71:0/1975207433 >> v1:172.21.15.71:6789/0 conn(0x7f51a414fed0 legacy=0x7f51a40c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a2d9d640 1 -- 172.21.15.71:0/1975207433 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a40bff50 con 0x7f51a4156740 2024-04-16T15:40:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.676+0000 7f51a1d9b640 1 --2- 172.21.15.71:0/1975207433 >> v2:172.21.15.122:3300/0 conn(0x7f51a40a8640 0x7f51a40bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.677+0000 7f51a2d9d640 1 --2- 172.21.15.71:0/1975207433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a40c8640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51a4063350 tx=0x7f519c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eca788fc2394d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.677+0000 7f51a159a640 1 -- 172.21.15.71:0/1975207433 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c035020 con 0x7f51a4156740 2024-04-16T15:40:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.677+0000 7f51a159a640 1 -- 172.21.15.71:0/1975207433 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519c031830 con 0x7f51a4156740 2024-04-16T15:40:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.677+0000 7f51a159a640 1 -- 172.21.15.71:0/1975207433 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c031c00 con 0x7f51a4156740 2024-04-16T15:40:27.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.678+0000 7f51a8f82640 1 -- 172.21.15.71:0/1975207433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 msgr2=0x7f51a40c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:27.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.678+0000 7f51a8f82640 1 --2- 172.21.15.71:0/1975207433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a40c8640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51a4063350 tx=0x7f519c02b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.678+0000 7f51a8f82640 1 -- 172.21.15.71:0/1975207433 shutdown_connections 2024-04-16T15:40:27.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.678+0000 7f51a8f82640 1 --2- 172.21.15.71:0/1975207433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a40c8640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.678+0000 7f51a8f82640 1 --2- 172.21.15.71:0/1975207433 >> v2:172.21.15.122:3300/0 conn(0x7f51a40a8640 0x7f51a40bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.678+0000 7f51a8f82640 1 -- 172.21.15.71:0/1975207433 >> 172.21.15.71:0/1975207433 conn(0x7f51a40c5e20 msgr2=0x7f51a40c6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:27.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.678+0000 7f51a8f82640 1 -- 172.21.15.71:0/1975207433 shutdown_connections 2024-04-16T15:40:27.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.678+0000 7f51a8f82640 1 -- 172.21.15.71:0/1975207433 wait complete. 2024-04-16T15:40:27.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.679+0000 7f51a8f82640 1 Processor -- start 2024-04-16T15:40:27.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.679+0000 7f51a8f82640 1 -- start start 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a8f82640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51a40a8640 0x7f51a41efab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a8f82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a414fed0 0x7f51a41e2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a8f82640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a4157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a8f82640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4156220 con 0x7f51a4156740 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a8f82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4155f40 con 0x7f51a414fed0 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a8f82640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f51a4156bb0 con 0x7f51a40a8640 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a4157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a4157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49408/0 (socket says 172.21.15.71:49408) 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 -- 172.21.15.71:0/2701397610 learned_addr learned my addr 172.21.15.71:0/2701397610 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a259c640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51a40a8640 0x7f51a41efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a1d9b640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a414fed0 0x7f51a41e2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:27.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 -- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51a40a8640 msgr2=0x7f51a41efab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51a40a8640 0x7f51a41efab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 -- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a414fed0 msgr2=0x7f51a41e2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a414fed0 0x7f51a41e2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 -- 172.21.15.71:0/2701397610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a4149c10 con 0x7f51a4156740 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a259c640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51a40a8640 0x7f51a41efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a1d9b640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a414fed0 0x7f51a41e2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.680+0000 7f51a2d9d640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a4157100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f519c02bee0 tx=0x7f519c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.681+0000 7f5192ffd640 1 -- 172.21.15.71:0/2701397610 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c041070 con 0x7f51a4156740 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.681+0000 7f5192ffd640 1 -- 172.21.15.71:0/2701397610 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f519c03a8d0 con 0x7f51a4156740 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.681+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a4149970 con 0x7f51a4156740 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.681+0000 7f5192ffd640 1 -- 172.21.15.71:0/2701397610 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c035040 con 0x7f51a4156740 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.681+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41e2e80 con 0x7f51a4156740 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.682+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a4068be0 con 0x7f51a4156740 2024-04-16T15:40:27.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.682+0000 7f5192ffd640 1 -- 172.21.15.71:0/2701397610 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f519c038020 con 0x7f51a4156740 2024-04-16T15:40:27.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.683+0000 7f5192ffd640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f518406c8f0 0x7f518406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:27.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.683+0000 7f5192ffd640 1 -- 172.21.15.71:0/2701397610 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f519c0ad8a0 con 0x7f51a4156740 2024-04-16T15:40:27.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.683+0000 7f51a259c640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f518406c8f0 0x7f518406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:27.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.683+0000 7f51a259c640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f518406c8f0 0x7f518406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f51940012e0 tx=0x7f5194007d90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:27.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.687+0000 7f5192ffd640 1 -- 172.21.15.71:0/2701397610 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f519c07ba40 con 0x7f51a4156740 2024-04-16T15:40:27.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:27.861+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f51a4149690 con 0x7f51a4156740 2024-04-16T15:40:28.344 INFO:tasks.ceph.mds.b.smithi122.stderr:2024-04-16T15:40:28.345+0000 7fa0d9c36640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-16T15:40:28.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.542+0000 7f5192ffd640 1 -- 172.21.15.71:0/2701397610 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f519c07b340 con 0x7f51a4156740 2024-04-16T15:40:28.541 INFO:teuthology.orchestra.run.smithi071.stderr:set pool 4 allow_ec_overwrites to true 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f518406c8f0 msgr2=0x7f518406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f518406c8f0 0x7f518406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f51940012e0 tx=0x7f5194007d90 comp rx=0 tx=0).stop 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 msgr2=0x7f51a4157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a4157100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f519c02bee0 tx=0x7f519c0046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 shutdown_connections 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f518406c8f0 0x7f518406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4156740 0x7f51a4157100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a414fed0 0x7f51a41e2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 --2- 172.21.15.71:0/2701397610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51a40a8640 0x7f51a41efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 >> 172.21.15.71:0/2701397610 conn(0x7f51a40c5e20 msgr2=0x7f51a414b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:28.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 shutdown_connections 2024-04-16T15:40:28.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.546+0000 7f51a8f82640 1 -- 172.21.15.71:0/2701397610 wait complete. 2024-04-16T15:40:28.572 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-16T15:40:28.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.680+0000 7fbacec35640 1 Processor -- start 2024-04-16T15:40:28.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.680+0000 7fbacec35640 1 -- start start 2024-04-16T15:40:28.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacec35640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fbac814da40 0x7fbac80ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:28.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacec35640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac80aca10 0x7fbac80c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:28.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacec35640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbac80c3fb0 0x7fbac80c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:28.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacec35640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fbac814de20 con 0x7fbac814da40 2024-04-16T15:40:28.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacec35640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fbac80c9120 con 0x7fbac80c3fb0 2024-04-16T15:40:28.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacec35640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbac81548b0 con 0x7fbac80aca10 2024-04-16T15:40:28.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacc9d1640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fbac814da40 0x7fbac80ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:28.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacc9d1640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fbac814da40 0x7fbac80ac4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49422/0 (socket says 172.21.15.71:49422) 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.681+0000 7fbacc9d1640 1 -- 172.21.15.71:0/2600353767 learned_addr learned my addr 172.21.15.71:0/2600353767 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbacd1d2640 1 --2- 172.21.15.71:0/2600353767 >> v2:172.21.15.122:3300/0 conn(0x7fbac80c3fb0 0x7fbac80c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbac7fff640 1 --2- 172.21.15.71:0/2600353767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac80aca10 0x7fbac80c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbacc9d1640 1 -- 172.21.15.71:0/2600353767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac80aca10 msgr2=0x7fbac80c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbacc9d1640 1 --2- 172.21.15.71:0/2600353767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac80aca10 0x7fbac80c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbacc9d1640 1 -- 172.21.15.71:0/2600353767 >> v2:172.21.15.122:3300/0 conn(0x7fbac80c3fb0 msgr2=0x7fbac80c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbacc9d1640 1 --2- 172.21.15.71:0/2600353767 >> v2:172.21.15.122:3300/0 conn(0x7fbac80c3fb0 0x7fbac80c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbacc9d1640 1 -- 172.21.15.71:0/2600353767 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac80ae840 con 0x7fbac814da40 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbac7fff640 1 --2- 172.21.15.71:0/2600353767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac80aca10 0x7fbac80c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:28.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbacc9d1640 1 --2- 172.21.15.71:0/2600353767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac814da40 0x7fbac80ac4d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbac80c4440 tx=0x7fbab802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c84dc32db407f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:28.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.682+0000 7fbac77fe640 1 -- 172.21.15.71:0/2600353767 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab802ca70 con 0x7fbac814da40 2024-04-16T15:40:28.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbac77fe640 1 -- 172.21.15.71:0/2600353767 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab802cc10 con 0x7fbac814da40 2024-04-16T15:40:28.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbac77fe640 1 -- 172.21.15.71:0/2600353767 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab80343f0 con 0x7fbac814da40 2024-04-16T15:40:28.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 -- 172.21.15.71:0/2600353767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac814da40 msgr2=0x7fbac80ac4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:28.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 --2- 172.21.15.71:0/2600353767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac814da40 0x7fbac80ac4d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbac80c4440 tx=0x7fbab802bed0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 -- 172.21.15.71:0/2600353767 shutdown_connections 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 --2- 172.21.15.71:0/2600353767 >> v2:172.21.15.122:3300/0 conn(0x7fbac80c3fb0 0x7fbac80c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 --2- 172.21.15.71:0/2600353767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac80aca10 0x7fbac80c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 --2- 172.21.15.71:0/2600353767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac814da40 0x7fbac80ac4d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 -- 172.21.15.71:0/2600353767 >> 172.21.15.71:0/2600353767 conn(0x7fbac80c9420 msgr2=0x7fbac80c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 -- 172.21.15.71:0/2600353767 shutdown_connections 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.683+0000 7fbacec35640 1 -- 172.21.15.71:0/2600353767 wait complete. 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.684+0000 7fbacec35640 1 Processor -- start 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.684+0000 7fbacec35640 1 -- start start 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbacec35640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac80aca10 0x7fbac80c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbacec35640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac80c3fb0 0x7fbac814a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbacec35640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac814da40 0x7fbac814ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:28.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbacec35640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbac80c4a10 con 0x7fbac80c3fb0 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbacec35640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbac80c73c0 con 0x7fbac80aca10 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbacec35640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbac80c39f0 con 0x7fbac814da40 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbac7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac80c3fb0 0x7fbac814a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbac7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac80c3fb0 0x7fbac814a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49436/0 (socket says 172.21.15.71:49436) 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbac7fff640 1 -- 172.21.15.71:0/20501818 learned_addr learned my addr 172.21.15.71:0/20501818 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbacd1d2640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac814da40 0x7fbac814ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbacc9d1640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac80aca10 0x7fbac80c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbac7fff640 1 -- 172.21.15.71:0/20501818 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac814da40 msgr2=0x7fbac814ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbac7fff640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac814da40 0x7fbac814ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbac7fff640 1 -- 172.21.15.71:0/20501818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac80aca10 msgr2=0x7fbac80c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.685+0000 7fbac7fff640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac80aca10 0x7fbac80c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.686+0000 7fbac7fff640 1 -- 172.21.15.71:0/20501818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac8154b60 con 0x7fbac80c3fb0 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.686+0000 7fbac7fff640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac80c3fb0 0x7fbac814a730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbab000c7f0 tx=0x7fbab000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.686+0000 7fbac57fa640 1 -- 172.21.15.71:0/20501818 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0015070 con 0x7fbac80c3fb0 2024-04-16T15:40:28.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.686+0000 7fbac57fa640 1 -- 172.21.15.71:0/20501818 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab0007820 con 0x7fbac80c3fb0 2024-04-16T15:40:28.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.686+0000 7fbac57fa640 1 -- 172.21.15.71:0/20501818 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0007c10 con 0x7fbac80c3fb0 2024-04-16T15:40:28.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.686+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac81ef880 con 0x7fbac80c3fb0 2024-04-16T15:40:28.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.686+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac80c5d50 con 0x7fbac80c3fb0 2024-04-16T15:40:28.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.687+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac8068be0 con 0x7fbac80c3fb0 2024-04-16T15:40:28.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.687+0000 7fbac57fa640 1 -- 172.21.15.71:0/20501818 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbab0018070 con 0x7fbac80c3fb0 2024-04-16T15:40:28.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.688+0000 7fbac57fa640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbaa806c8f0 0x7fbaa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:28.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.688+0000 7fbac57fa640 1 -- 172.21.15.71:0/20501818 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fbab008bf10 con 0x7fbac80c3fb0 2024-04-16T15:40:28.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.688+0000 7fbacc9d1640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbaa806c8f0 0x7fbaa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:28.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.688+0000 7fbacc9d1640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbaa806c8f0 0x7fbaa806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbab8035000 tx=0x7fbab8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:28.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.692+0000 7fbac57fa640 1 -- 172.21.15.71:0/20501818 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbab0059950 con 0x7fbac80c3fb0 2024-04-16T15:40:28.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:28.914+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fbac80c8e90 con 0x7fbac80c3fb0 2024-04-16T15:40:29.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.563+0000 7fbac57fa640 1 -- 172.21.15.71:0/20501818 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fbab005dc50 con 0x7fbac80c3fb0 2024-04-16T15:40:29.562 INFO:teuthology.orchestra.run.smithi071.stderr:added data pool 4 to fsmap 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbaa806c8f0 msgr2=0x7fbaa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbaa806c8f0 0x7fbaa806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbab8035000 tx=0x7fbab8030000 comp rx=0 tx=0).stop 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac80c3fb0 msgr2=0x7fbac814a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac80c3fb0 0x7fbac814a730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbab000c7f0 tx=0x7fbab000ccb0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 shutdown_connections 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbaa806c8f0 0x7fbaa806edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbac814da40 0x7fbac814ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbac80c3fb0 0x7fbac814a730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 --2- 172.21.15.71:0/20501818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac80aca10 0x7fbac80c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 >> 172.21.15.71:0/20501818 conn(0x7fbac80c9420 msgr2=0x7fbac8153660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 shutdown_connections 2024-04-16T15:40:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.567+0000 7fbacec35640 1 -- 172.21.15.71:0/20501818 wait complete. 2024-04-16T15:40:29.579 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:40:29.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.689+0000 7f4b640f2640 1 Processor -- start 2024-04-16T15:40:29.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.689+0000 7f4b640f2640 1 -- start start 2024-04-16T15:40:29.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b640f2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c153450 0x7f4b5c0c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:29.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b640f2640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5c06e820 con 0x7f4b5c0c94c0 2024-04-16T15:40:29.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b640f2640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5c159fb0 con 0x7f4b5c0c99d0 2024-04-16T15:40:29.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b640f2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5c0c8150 con 0x7f4b5c153450 2024-04-16T15:40:29.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b6268f640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f4b5c0c94c0 0x7f4b5c1564c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56026/0 (socket says 172.21.15.71:56026) 2024-04-16T15:40:29.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b6268f640 1 -- 172.21.15.71:0/2865166077 learned_addr learned my addr 172.21.15.71:0/2865166077 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b61e8e640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c153450 0x7f4b5c0c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1174944399 0 0) 0x7f4b5c06e820 con 0x7f4b5c0c94c0 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b48003520 con 0x7f4b5c0c94c0 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463829112 0 0) 0x7f4b5c159fb0 con 0x7f4b5c0c99d0 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b5c06e820 con 0x7f4b5c0c99d0 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3716066676 0 0) 0x7f4b48003520 con 0x7f4b5c0c94c0 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c153450 msgr2=0x7f4b5c0c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c153450 0x7f4b5c0c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 >> v1:172.21.15.122:6789/0 conn(0x7f4b5c0c99d0 legacy=0x7f4b5c155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c0c8430 con 0x7f4b5c0c94c0 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f4b540019f0 con 0x7f4b5c0c94c0 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 >> v1:172.21.15.71:6789/0 conn(0x7f4b5c0c94c0 legacy=0x7f4b5c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b48003b70 0x7f4b48003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:29.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c153450 0x7f4b48005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c0c8150 con 0x7f4b48004800 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4800bb80 con 0x7f4b48003b70 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.690+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4800bd80 con 0x7f4b5c153450 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.691+0000 7f4b6168d640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.691+0000 7f4b61e8e640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b48003b70 0x7f4b48003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.691+0000 7f4b6268f640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c153450 0x7f4b48005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.691+0000 7f4b6168d640 1 -- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c153450 msgr2=0x7f4b48005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.691+0000 7f4b6168d640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c153450 0x7f4b48005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.691+0000 7f4b6168d640 1 -- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b48003b70 msgr2=0x7f4b48003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.691+0000 7f4b6168d640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b48003b70 0x7f4b48003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.691+0000 7f4b6168d640 1 -- 172.21.15.71:0/2865166077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b48003520 con 0x7f4b48004800 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b6168d640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b48004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0018d0 tx=0x7f4b48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f818efb9f9e54a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c037070 con 0x7f4b48004800 2024-04-16T15:40:29.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4c0052d0 con 0x7f4b48004800 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b60e8c640 1 -- 172.21.15.71:0/2865166077 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0014f0 con 0x7f4b48004800 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b640f2640 1 -- 172.21.15.71:0/2865166077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 msgr2=0x7f4b48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b640f2640 1 --2- 172.21.15.71:0/2865166077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b48004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0018d0 tx=0x7f4b48005830 comp rx=0 tx=0).stop 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b6268f640 1 -- 172.21.15.71:0/2865166077 reap_dead start 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b640f2640 1 -- 172.21.15.71:0/2865166077 shutdown_connections 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.692+0000 7f4b640f2640 1 -- 172.21.15.71:0/2865166077 >> 172.21.15.71:0/2865166077 conn(0x7f4b5c0ad2f0 msgr2=0x7f4b5c0ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.693+0000 7f4b640f2640 1 -- 172.21.15.71:0/2865166077 shutdown_connections 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.693+0000 7f4b640f2640 1 -- 172.21.15.71:0/2865166077 wait complete. 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b640f2640 1 Processor -- start 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b640f2640 1 -- start start 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b640f2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b5c0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b640f2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c1595e0 0x7f4b5c0c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:29.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b640f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b5c1eb3b0 0x7f4b5c1eb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b640f2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c0c60c0 con 0x7f4b48004800 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b640f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c0c5f40 con 0x7f4b5c1eb3b0 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b640f2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5c0c6240 con 0x7f4b5c1595e0 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b61e8e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b5c0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b61e8e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b5c0b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49444/0 (socket says 172.21.15.71:49444) 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.694+0000 7f4b61e8e640 1 -- 172.21.15.71:0/901480453 learned_addr learned my addr 172.21.15.71:0/901480453 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b6268f640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b5c1eb3b0 0x7f4b5c1eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b6168d640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c1595e0 0x7f4b5c0c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b61e8e640 1 -- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c1595e0 msgr2=0x7f4b5c0c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b61e8e640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c1595e0 0x7f4b5c0c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b61e8e640 1 -- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b5c1eb3b0 msgr2=0x7f4b5c1eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b61e8e640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b5c1eb3b0 0x7f4b5c1eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b61e8e640 1 -- 172.21.15.71:0/901480453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c158500 con 0x7f4b48004800 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b6168d640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c1595e0 0x7f4b5c0c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:29.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b61e8e640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b5c0b0e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b580119a0 tx=0x7f4b58011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:29.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b527fc640 1 -- 172.21.15.71:0/901480453 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800faf0 con 0x7f4b48004800 2024-04-16T15:40:29.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b527fc640 1 -- 172.21.15.71:0/901480453 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5800fc90 con 0x7f4b48004800 2024-04-16T15:40:29.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c150f90 con 0x7f4b48004800 2024-04-16T15:40:29.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b527fc640 1 -- 172.21.15.71:0/901480453 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58014440 con 0x7f4b48004800 2024-04-16T15:40:29.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.695+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c158a80 con 0x7f4b48004800 2024-04-16T15:40:29.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.696+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c074060 con 0x7f4b48004800 2024-04-16T15:40:29.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.697+0000 7f4b527fc640 1 -- 172.21.15.71:0/901480453 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b58012040 con 0x7f4b48004800 2024-04-16T15:40:29.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.697+0000 7f4b527fc640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4b4006c920 0x7f4b4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:29.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.697+0000 7f4b6168d640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4b4006c920 0x7f4b4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:29.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.697+0000 7f4b527fc640 1 -- 172.21.15.71:0/901480453 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4b5808c330 con 0x7f4b48004800 2024-04-16T15:40:29.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.697+0000 7f4b6168d640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4b4006c920 0x7f4b4006ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c034000 tx=0x7f4b4c031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:29.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.701+0000 7f4b527fc640 1 -- 172.21.15.71:0/901480453 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b58019030 con 0x7f4b48004800 2024-04-16T15:40:29.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.913+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b5c06e4c0 con 0x7f4b48004800 2024-04-16T15:40:29.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.914+0000 7f4b527fc640 1 -- 172.21.15.71:0/901480453 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7f4b5805a4b0 con 0x7f4b48004800 2024-04-16T15:40:29.913 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:29.913 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:28.915544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:40:29.913 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 7 2024-04-16T15:40:29.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4b4006c920 msgr2=0x7f4b4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4b4006c920 0x7f4b4006ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c034000 tx=0x7f4b4c031000 comp rx=0 tx=0).stop 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 msgr2=0x7f4b5c0b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b5c0b0e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b580119a0 tx=0x7f4b58011e60 comp rx=0 tx=0).stop 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 shutdown_connections 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4b4006c920 0x7f4b4006ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b5c1eb3b0 0x7f4b5c1eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b5c1595e0 0x7f4b5c0c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 --2- 172.21.15.71:0/901480453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b48004800 0x7f4b5c0b0e80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.915+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 >> 172.21.15.71:0/901480453 conn(0x7f4b5c0ad2f0 msgr2=0x7f4b5c06d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.916+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 shutdown_connections 2024-04-16T15:40:29.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:29.916+0000 7f4b640f2640 1 -- 172.21.15.71:0/901480453 wait complete. 2024-04-16T15:40:29.926 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:30.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.029+0000 7f25f8c7b640 1 Processor -- start 2024-04-16T15:40:30.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.029+0000 7f25f8c7b640 1 -- start start 2024-04-16T15:40:30.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f8c7b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f25f4153570 0x7f25f40c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f8c7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f40c4350 0x7f25f40c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f8c7b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f25f40c62e0 0x7f25f4159740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f8c7b640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f25f40c41a0 con 0x7f25f40c62e0 2024-04-16T15:40:30.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f8c7b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f25f40c82d0 con 0x7f25f4153570 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f8c7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f25f4155d70 con 0x7f25f40c4350 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f2d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f25f40c62e0 0x7f25f4159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f2d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f25f40c62e0 0x7f25f4159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49446/0 (socket says 172.21.15.71:49446) 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.030+0000 7f25f2d9d640 1 -- 172.21.15.71:0/4041109334 learned_addr learned my addr 172.21.15.71:0/4041109334 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f1d9b640 1 --2- 172.21.15.71:0/4041109334 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f40c4350 0x7f25f40c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f259c640 1 --2- 172.21.15.71:0/4041109334 >> v2:172.21.15.122:3300/0 conn(0x7f25f4153570 0x7f25f40c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f2d9d640 1 -- 172.21.15.71:0/4041109334 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f40c4350 msgr2=0x7f25f40c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f2d9d640 1 --2- 172.21.15.71:0/4041109334 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f40c4350 0x7f25f40c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f2d9d640 1 -- 172.21.15.71:0/4041109334 >> v2:172.21.15.122:3300/0 conn(0x7f25f4153570 msgr2=0x7f25f40c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f2d9d640 1 --2- 172.21.15.71:0/4041109334 >> v2:172.21.15.122:3300/0 conn(0x7f25f4153570 0x7f25f40c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f2d9d640 1 -- 172.21.15.71:0/4041109334 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f40c8d50 con 0x7f25f40c62e0 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f1d9b640 1 --2- 172.21.15.71:0/4041109334 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f40c4350 0x7f25f40c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f259c640 1 --2- 172.21.15.71:0/4041109334 >> v2:172.21.15.122:3300/0 conn(0x7f25f4153570 0x7f25f40c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.031+0000 7f25f2d9d640 1 --2- 172.21.15.71:0/4041109334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f4159740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f25f40c3d20 tx=0x7f25ec02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecba38ebb91f32d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f159a640 1 -- 172.21.15.71:0/4041109334 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec02c950 con 0x7f25f40c62e0 2024-04-16T15:40:30.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f159a640 1 -- 172.21.15.71:0/4041109334 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec02caf0 con 0x7f25f40c62e0 2024-04-16T15:40:30.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f159a640 1 -- 172.21.15.71:0/4041109334 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec0343f0 con 0x7f25f40c62e0 2024-04-16T15:40:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4041109334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 msgr2=0x7f25f4159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4041109334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f4159740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f25f40c3d20 tx=0x7f25ec02bdb0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4041109334 shutdown_connections 2024-04-16T15:40:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4041109334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f4159740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4041109334 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f40c4350 0x7f25f40c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4041109334 >> v2:172.21.15.122:3300/0 conn(0x7f25f4153570 0x7f25f40c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4041109334 >> 172.21.15.71:0/4041109334 conn(0x7f25f40c96e0 msgr2=0x7f25f40c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4041109334 shutdown_connections 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.032+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4041109334 wait complete. 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f8c7b640 1 Processor -- start 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f8c7b640 1 -- start start 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f8c7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f40c4350 0x7f25f41eb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f8c7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f8c7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f4153570 0x7f25f4158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f8c7b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f25f4159c80 con 0x7f25f40c62e0 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f8c7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25f40c8ff0 con 0x7f25f40c4350 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f8c7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f25f4159e00 con 0x7f25f4153570 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f1d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f1d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49450/0 (socket says 172.21.15.71:49450) 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f1d9b640 1 -- 172.21.15.71:0/4218092716 learned_addr learned my addr 172.21.15.71:0/4218092716 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f2d9d640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f4153570 0x7f25f4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.034+0000 7f25f259c640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f40c4350 0x7f25f41eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25f1d9b640 1 -- 172.21.15.71:0/4218092716 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f4153570 msgr2=0x7f25f4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25f1d9b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f4153570 0x7f25f4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25f1d9b640 1 -- 172.21.15.71:0/4218092716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f40c4350 msgr2=0x7f25f41eb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25f1d9b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f40c4350 0x7f25f41eb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25f1d9b640 1 -- 172.21.15.71:0/4218092716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f40aa530 con 0x7f25f40c62e0 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25f1d9b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f414df20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25e800e810 tx=0x7f25e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25e2ffd640 1 -- 172.21.15.71:0/4218092716 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e800f040 con 0x7f25f40c62e0 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25e2ffd640 1 -- 172.21.15.71:0/4218092716 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e8004030 con 0x7f25f40c62e0 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f41ebcf0 con 0x7f25f40c62e0 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25e2ffd640 1 -- 172.21.15.71:0/4218092716 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8004400 con 0x7f25f40c62e0 2024-04-16T15:40:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.035+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f41ec2b0 con 0x7f25f40c62e0 2024-04-16T15:40:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.036+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f4068d20 con 0x7f25f40c62e0 2024-04-16T15:40:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.036+0000 7f25e2ffd640 1 -- 172.21.15.71:0/4218092716 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25e800c6d0 con 0x7f25f40c62e0 2024-04-16T15:40:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.037+0000 7f25e2ffd640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f25d406c8f0 0x7f25d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.037+0000 7f25f259c640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f25d406c8f0 0x7f25d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.037+0000 7f25e2ffd640 1 -- 172.21.15.71:0/4218092716 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f25e808b9d0 con 0x7f25f40c62e0 2024-04-16T15:40:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.037+0000 7f25f259c640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f25d406c8f0 0x7f25d406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25f414e6e0 tx=0x7f25e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.041+0000 7f25e2ffd640 1 -- 172.21.15.71:0/4218092716 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25e8017020 con 0x7f25f40c62e0 2024-04-16T15:40:30.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.216+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25f40c72d0 con 0x7f25f40c62e0 2024-04-16T15:40:30.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.218+0000 7f25e2ffd640 1 -- 172.21.15.71:0/4218092716 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f25e8059500 con 0x7f25f40c62e0 2024-04-16T15:40:30.217 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:30.217 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f25d406c8f0 msgr2=0x7f25d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f25d406c8f0 0x7f25d406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25f414e6e0 tx=0x7f25e4008040 comp rx=0 tx=0).stop 2024-04-16T15:40:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 msgr2=0x7f25f414df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f414df20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25e800e810 tx=0x7f25e800ecd0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 shutdown_connections 2024-04-16T15:40:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f25d406c8f0 0x7f25d406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25f4153570 0x7f25f4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f40c62e0 0x7f25f414df20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 --2- 172.21.15.71:0/4218092716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25f40c4350 0x7f25f41eb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 >> 172.21.15.71:0/4218092716 conn(0x7f25f40c96e0 msgr2=0x7f25f41514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:30.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 shutdown_connections 2024-04-16T15:40:30.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.220+0000 7f25f8c7b640 1 -- 172.21.15.71:0/4218092716 wait complete. 2024-04-16T15:40:30.230 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:30.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.334+0000 7f2a391a8640 1 Processor -- start 2024-04-16T15:40:30.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.334+0000 7f2a391a8640 1 -- start start 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a391a8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2a34153570 0x7f2a340c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a391a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c4350 0x7f2a340c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a391a8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a340c82d0 con 0x7f2a340c62e0 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a391a8640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f2a340c41a0 con 0x7f2a34153570 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a391a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2a34155d70 con 0x7f2a340c4350 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a32d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2a34153570 0x7f2a340c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a32d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2a34153570 0x7f2a340c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49466/0 (socket says 172.21.15.71:49466) 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a3359e640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f2a340c62e0 0x7f2a34159740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56032/0 (socket says 172.21.15.71:56032) 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a32d9d640 1 -- 172.21.15.71:0/2480284248 learned_addr learned my addr 172.21.15.71:0/2480284248 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a3259c640 1 --2- 172.21.15.71:0/2480284248 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c4350 0x7f2a340c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a32d9d640 1 -- 172.21.15.71:0/2480284248 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c4350 msgr2=0x7f2a340c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a32d9d640 1 --2- 172.21.15.71:0/2480284248 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c4350 0x7f2a340c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a32d9d640 1 -- 172.21.15.71:0/2480284248 >> v1:172.21.15.71:6789/0 conn(0x7f2a340c62e0 legacy=0x7f2a34159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a32d9d640 1 -- 172.21.15.71:0/2480284248 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a340c8d50 con 0x7f2a34153570 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a31d9b640 1 -- 172.21.15.71:0/2480284248 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 745268279 0 0) 0x7f2a340c82d0 con 0x7f2a340c62e0 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.335+0000 7f2a3259c640 1 --2- 172.21.15.71:0/2480284248 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c4350 0x7f2a340c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.336+0000 7f2a32d9d640 1 --2- 172.21.15.71:0/2480284248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a34153570 0x7f2a340c52b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009870 tx=0x7f2a2402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdf0ed4fb130391 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.336+0000 7f2a31d9b640 1 -- 172.21.15.71:0/2480284248 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240317c0 con 0x7f2a34153570 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.336+0000 7f2a31d9b640 1 -- 172.21.15.71:0/2480284248 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24037070 con 0x7f2a34153570 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.336+0000 7f2a31d9b640 1 -- 172.21.15.71:0/2480284248 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24031b80 con 0x7f2a34153570 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.337+0000 7f2a391a8640 1 -- 172.21.15.71:0/2480284248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a34153570 msgr2=0x7f2a340c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.337+0000 7f2a391a8640 1 --2- 172.21.15.71:0/2480284248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a34153570 0x7f2a340c52b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009870 tx=0x7f2a2402b860 comp rx=0 tx=0).stop 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.337+0000 7f2a391a8640 1 -- 172.21.15.71:0/2480284248 shutdown_connections 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.337+0000 7f2a391a8640 1 --2- 172.21.15.71:0/2480284248 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c4350 0x7f2a340c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.337+0000 7f2a391a8640 1 --2- 172.21.15.71:0/2480284248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a34153570 0x7f2a340c52b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.337+0000 7f2a391a8640 1 -- 172.21.15.71:0/2480284248 >> 172.21.15.71:0/2480284248 conn(0x7f2a340c96e0 msgr2=0x7f2a340c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.337+0000 7f2a391a8640 1 -- 172.21.15.71:0/2480284248 shutdown_connections 2024-04-16T15:40:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.337+0000 7f2a391a8640 1 -- 172.21.15.71:0/2480284248 wait complete. 2024-04-16T15:40:30.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.338+0000 7f2a391a8640 1 Processor -- start 2024-04-16T15:40:30.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.338+0000 7f2a391a8640 1 -- start start 2024-04-16T15:40:30.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a391a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a340c4350 0x7f2a3406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a391a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c62e0 0x7f2a340c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a391a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a34153570 0x7f2a341e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a391a8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34159e50 con 0x7f2a340c4350 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a391a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a340c8ff0 con 0x7f2a34153570 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a391a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2a34159fd0 con 0x7f2a340c62e0 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a340c4350 0x7f2a3406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a340c4350 0x7f2a3406d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49476/0 (socket says 172.21.15.71:49476) 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 -- 172.21.15.71:0/3794241088 learned_addr learned my addr 172.21.15.71:0/3794241088 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a3259c640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c62e0 0x7f2a340c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a3359e640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a34153570 0x7f2a341e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 -- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c62e0 msgr2=0x7f2a340c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c62e0 0x7f2a340c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 -- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a34153570 msgr2=0x7f2a341e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a34153570 0x7f2a341e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 -- 172.21.15.71:0/3794241088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3406e130 con 0x7f2a340c4350 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a3259c640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c62e0 0x7f2a340c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a32d9d640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a340c4350 0x7f2a3406d450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2a24000c00 tx=0x7f2a240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.339+0000 7f2a1b7fe640 1 -- 172.21.15.71:0/3794241088 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24032860 con 0x7f2a340c4350 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.340+0000 7f2a1b7fe640 1 -- 172.21.15.71:0/3794241088 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24032a00 con 0x7f2a340c4350 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.340+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3406e410 con 0x7f2a340c4350 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.340+0000 7f2a1b7fe640 1 -- 172.21.15.71:0/3794241088 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24032df0 con 0x7f2a340c4350 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.340+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a341ec9a0 con 0x7f2a340c4350 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.340+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a341ecba0 con 0x7f2a340c4350 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.341+0000 7f2a1b7fe640 1 -- 172.21.15.71:0/3794241088 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a2404c060 con 0x7f2a340c4350 2024-04-16T15:40:30.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.341+0000 7f2a1b7fe640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2a2006c8f0 0x7f2a2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.342+0000 7f2a1b7fe640 1 -- 172.21.15.71:0/3794241088 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2a240ad8a0 con 0x7f2a340c4350 2024-04-16T15:40:30.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.342+0000 7f2a3259c640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2a2006c8f0 0x7f2a2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.342+0000 7f2a3259c640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2a2006c8f0 0x7f2a2006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a34153270 tx=0x7f2a28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.346+0000 7f2a1b7fe640 1 -- 172.21.15.71:0/3794241088 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a24040070 con 0x7f2a340c4350 2024-04-16T15:40:30.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.523+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a340c7c50 con 0x7f2a340c4350 2024-04-16T15:40:30.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.524+0000 7f2a1b7fe640 1 -- 172.21.15.71:0/3794241088 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f2a2407b4f0 con 0x7f2a340c4350 2024-04-16T15:40:30.523 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:30.523 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:30.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2a2006c8f0 msgr2=0x7f2a2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2a2006c8f0 0x7f2a2006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a34153270 tx=0x7f2a28008040 comp rx=0 tx=0).stop 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a340c4350 msgr2=0x7f2a3406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a340c4350 0x7f2a3406d450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2a24000c00 tx=0x7f2a240046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 shutdown_connections 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2a2006c8f0 0x7f2a2006edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a34153570 0x7f2a341e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2a340c62e0 0x7f2a340c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 --2- 172.21.15.71:0/3794241088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a340c4350 0x7f2a3406d450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.526+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 >> 172.21.15.71:0/3794241088 conn(0x7f2a340c96e0 msgr2=0x7f2a340c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.527+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 shutdown_connections 2024-04-16T15:40:30.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.527+0000 7f2a391a8640 1 -- 172.21.15.71:0/3794241088 wait complete. 2024-04-16T15:40:30.552 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:40:30.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.656+0000 7feed1b7c640 1 Processor -- start 2024-04-16T15:40:30.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.656+0000 7feed1b7c640 1 -- start start 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feed1b7c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7feecc0c8ee0 0x7feecc0accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feed1b7c640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc1549a0 con 0x7feecc0c8130 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feed1b7c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc0c7650 con 0x7feecc0713a0 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feed1b7c640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7feecc0c7400 con 0x7feecc0c8ee0 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feecbfff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7feecc0c8ee0 0x7feecc0accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feecaffd640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7feecc0c8130 0x7feecc0ac5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56036/0 (socket says 172.21.15.71:56036) 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feecaffd640 1 -- 172.21.15.71:0/3558953106 learned_addr learned my addr 172.21.15.71:0/3558953106 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feecbfff640 1 -- 172.21.15.71:0/3558953106 >> v1:172.21.15.122:6789/0 conn(0x7feecc0713a0 legacy=0x7feecc155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feecbfff640 1 -- 172.21.15.71:0/3558953106 >> v1:172.21.15.71:6789/0 conn(0x7feecc0c8130 legacy=0x7feecc0ac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.657+0000 7feecbfff640 1 -- 172.21.15.71:0/3558953106 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc0c7930 con 0x7feecc0c8ee0 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feecbfff640 1 --2- 172.21.15.71:0/3558953106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0c8ee0 0x7feecc0accc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feec4009d90 tx=0x7feec402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb2122e08c8d82c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feeca7fc640 1 -- 172.21.15.71:0/3558953106 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4039070 con 0x7feecc0c8ee0 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feeca7fc640 1 -- 172.21.15.71:0/3558953106 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec402cbd0 con 0x7feecc0c8ee0 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feeca7fc640 1 -- 172.21.15.71:0/3558953106 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4034440 con 0x7feecc0c8ee0 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feed1b7c640 1 -- 172.21.15.71:0/3558953106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0c8ee0 msgr2=0x7feecc0accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feed1b7c640 1 --2- 172.21.15.71:0/3558953106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0c8ee0 0x7feecc0accc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feec4009d90 tx=0x7feec402be20 comp rx=0 tx=0).stop 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feed1b7c640 1 -- 172.21.15.71:0/3558953106 shutdown_connections 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feed1b7c640 1 --2- 172.21.15.71:0/3558953106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0c8ee0 0x7feecc0accc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feed1b7c640 1 -- 172.21.15.71:0/3558953106 >> 172.21.15.71:0/3558953106 conn(0x7feecc0c96e0 msgr2=0x7feecc0c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feed1b7c640 1 -- 172.21.15.71:0/3558953106 shutdown_connections 2024-04-16T15:40:30.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.658+0000 7feed1b7c640 1 -- 172.21.15.71:0/3558953106 wait complete. 2024-04-16T15:40:30.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.659+0000 7feed1b7c640 1 Processor -- start 2024-04-16T15:40:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feed1b7c640 1 -- start start 2024-04-16T15:40:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feed1b7c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0713a0 0x7feecc0aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feed1b7c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feecc0c8130 0x7feecc1e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feed1b7c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feecc0c8ee0 0x7feecc1ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feed1b7c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feecc071780 con 0x7feecc0713a0 2024-04-16T15:40:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feed1b7c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feecc0c7ca0 con 0x7feecc0c8ee0 2024-04-16T15:40:30.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feed1b7c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feecc071900 con 0x7feecc0c8130 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feecb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0713a0 0x7feecc0aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feecb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0713a0 0x7feecc0aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49490/0 (socket says 172.21.15.71:49490) 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.660+0000 7feecb7fe640 1 -- 172.21.15.71:0/529266871 learned_addr learned my addr 172.21.15.71:0/529266871 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecb7fe640 1 -- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feecc0c8130 msgr2=0x7feecc1e9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecaffd640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feecc0c8130 0x7feecc1e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecbfff640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feecc0c8ee0 0x7feecc1ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecb7fe640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feecc0c8130 0x7feecc1e9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecb7fe640 1 -- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feecc0c8ee0 msgr2=0x7feecc1ea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecb7fe640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feecc0c8ee0 0x7feecc1ea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecb7fe640 1 -- 172.21.15.71:0/529266871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc153910 con 0x7feecc0713a0 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecbfff640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feecc0c8ee0 0x7feecc1ea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecaffd640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feecc0c8130 0x7feecc1e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feecb7fe640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0713a0 0x7feecc0aa4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feebc00b3f0 tx=0x7feebc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feeb3fff640 1 -- 172.21.15.71:0/529266871 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc015020 con 0x7feecc0713a0 2024-04-16T15:40:30.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feeb3fff640 1 -- 172.21.15.71:0/529266871 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc011830 con 0x7feecc0713a0 2024-04-16T15:40:30.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feeb3fff640 1 -- 172.21.15.71:0/529266871 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc011c20 con 0x7feecc0713a0 2024-04-16T15:40:30.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc0c42c0 con 0x7feecc0713a0 2024-04-16T15:40:30.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.661+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc14c950 con 0x7feecc0713a0 2024-04-16T15:40:30.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.662+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feecc068d20 con 0x7feecc0713a0 2024-04-16T15:40:30.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.663+0000 7feeb3fff640 1 -- 172.21.15.71:0/529266871 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feebc018020 con 0x7feecc0713a0 2024-04-16T15:40:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.663+0000 7feeb3fff640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feeb806c8f0 0x7feeb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.663+0000 7feecaffd640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feeb806c8f0 0x7feeb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.663+0000 7feeb3fff640 1 -- 172.21.15.71:0/529266871 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feebc08baa0 con 0x7feecc0713a0 2024-04-16T15:40:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.664+0000 7feecaffd640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feeb806c8f0 0x7feeb806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feecc0c37a0 tx=0x7feec0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.668+0000 7feeb3fff640 1 -- 172.21.15.71:0/529266871 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feebc01f030 con 0x7feecc0713a0 2024-04-16T15:40:30.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.878+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feecc156190 con 0x7feecc0713a0 2024-04-16T15:40:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.878+0000 7feeb3fff640 1 -- 172.21.15.71:0/529266871 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7feebc0172c0 con 0x7feecc0713a0 2024-04-16T15:40:30.877 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:30.877 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:28.915544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:40:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 7 2024-04-16T15:40:30.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feeb806c8f0 msgr2=0x7feeb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feeb806c8f0 0x7feeb806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feecc0c37a0 tx=0x7feec0009040 comp rx=0 tx=0).stop 2024-04-16T15:40:30.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0713a0 msgr2=0x7feecc0aa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0713a0 0x7feecc0aa4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feebc00b3f0 tx=0x7feebc00b8b0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 shutdown_connections 2024-04-16T15:40:30.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feeb806c8f0 0x7feeb806edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feecc0c8ee0 0x7feecc1ea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feecc0c8130 0x7feecc1e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 --2- 172.21.15.71:0/529266871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feecc0713a0 0x7feecc0aa4e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 >> 172.21.15.71:0/529266871 conn(0x7feecc0c96e0 msgr2=0x7feecc0c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:30.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 shutdown_connections 2024-04-16T15:40:30.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.880+0000 7feed1b7c640 1 -- 172.21.15.71:0/529266871 wait complete. 2024-04-16T15:40:30.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:30.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.994+0000 7f8179e0d640 1 Processor -- start 2024-04-16T15:40:30.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.994+0000 7f8179e0d640 1 -- start start 2024-04-16T15:40:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8179e0d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8174153450 0x7f81740c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8179e0d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f81740c94c0 0x7f81741564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8179e0d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f817406e820 con 0x7f81740c99d0 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8179e0d640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f8174159fb0 con 0x7f81740c94c0 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8179e0d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f81740c8150 con 0x7f8174153450 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8173fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f81740c94c0 0x7f81741564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8173fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f81740c94c0 0x7f81741564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49492/0 (socket says 172.21.15.71:49492) 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8173fff640 1 -- 172.21.15.71:0/3268368133 learned_addr learned my addr 172.21.15.71:0/3268368133 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f81737fe640 1 --2- 172.21.15.71:0/3268368133 >> v2:172.21.15.122:3300/0 conn(0x7f8174153450 0x7f81740c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8173fff640 1 -- 172.21.15.71:0/3268368133 >> v2:172.21.15.122:3300/0 conn(0x7f8174153450 msgr2=0x7f81740c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8173fff640 1 --2- 172.21.15.71:0/3268368133 >> v2:172.21.15.122:3300/0 conn(0x7f8174153450 0x7f81740c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8173fff640 1 -- 172.21.15.71:0/3268368133 >> v1:172.21.15.122:6789/0 conn(0x7f81740c99d0 legacy=0x7f8174155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.995+0000 7f8173fff640 1 -- 172.21.15.71:0/3268368133 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81740c8430 con 0x7f81740c94c0 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.996+0000 7f8173fff640 1 --2- 172.21.15.71:0/3268368133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c94c0 0x7f81741564c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f817408d080 tx=0x7f816c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222abbd2be77a8b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.996+0000 7f81727fc640 1 -- 172.21.15.71:0/3268368133 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c02c9f0 con 0x7f81740c94c0 2024-04-16T15:40:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.996+0000 7f81727fc640 1 -- 172.21.15.71:0/3268368133 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f816c02cb90 con 0x7f81740c94c0 2024-04-16T15:40:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.996+0000 7f81727fc640 1 -- 172.21.15.71:0/3268368133 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c0343f0 con 0x7f81740c94c0 2024-04-16T15:40:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.997+0000 7f8179e0d640 1 -- 172.21.15.71:0/3268368133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c94c0 msgr2=0x7f81741564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.997+0000 7f8179e0d640 1 --2- 172.21.15.71:0/3268368133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c94c0 0x7f81741564c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f817408d080 tx=0x7f816c02be50 comp rx=0 tx=0).stop 2024-04-16T15:40:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.997+0000 7f8179e0d640 1 -- 172.21.15.71:0/3268368133 shutdown_connections 2024-04-16T15:40:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.997+0000 7f8179e0d640 1 --2- 172.21.15.71:0/3268368133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c94c0 0x7f81741564c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.997+0000 7f8179e0d640 1 --2- 172.21.15.71:0/3268368133 >> v2:172.21.15.122:3300/0 conn(0x7f8174153450 0x7f81740c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.997+0000 7f8179e0d640 1 -- 172.21.15.71:0/3268368133 >> 172.21.15.71:0/3268368133 conn(0x7f81740ad2f0 msgr2=0x7f81740ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.997+0000 7f8179e0d640 1 -- 172.21.15.71:0/3268368133 shutdown_connections 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.997+0000 7f8179e0d640 1 -- 172.21.15.71:0/3268368133 wait complete. 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.998+0000 7f8179e0d640 1 Processor -- start 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.998+0000 7f8179e0d640 1 -- start start 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8179e0d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f81740c94c0 0x7f81740ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8179e0d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c99d0 0x7f8174158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8179e0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8174153450 0x7f8174157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8179e0d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f81740c6480 con 0x7f81740c99d0 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8179e0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8174159c60 con 0x7f8174153450 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8179e0d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f81740c6600 con 0x7f81740c94c0 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c99d0 0x7f8174158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c99d0 0x7f8174158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49508/0 (socket says 172.21.15.71:49508) 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 -- 172.21.15.71:0/4076368410 learned_addr learned my addr 172.21.15.71:0/4076368410 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8173fff640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8174153450 0x7f8174157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f81737fe640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f81740c94c0 0x7f81740ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 -- 172.21.15.71:0/4076368410 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f81740c94c0 msgr2=0x7f81740ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f81740c94c0 0x7f81740ab940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 -- 172.21.15.71:0/4076368410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8174153450 msgr2=0x7f8174157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8174153450 0x7f8174157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 -- 172.21.15.71:0/4076368410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817414d6f0 con 0x7f81740c99d0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8172ffd640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c99d0 0x7f8174158a70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f816800c470 tx=0x7f816800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f815bfff640 1 -- 172.21.15.71:0/4076368410 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81680052d0 con 0x7f81740c99d0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f815bfff640 1 -- 172.21.15.71:0/4076368410 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168005490 con 0x7f81740c99d0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817414d9d0 con 0x7f81740c99d0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f815bfff640 1 -- 172.21.15.71:0/4076368410 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168007260 con 0x7f81740c99d0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:30.999+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817414eaa0 con 0x7f81740c99d0 2024-04-16T15:40:31.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.000+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8174074060 con 0x7f81740c99d0 2024-04-16T15:40:31.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.001+0000 7f815bfff640 1 -- 172.21.15.71:0/4076368410 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8168007400 con 0x7f81740c99d0 2024-04-16T15:40:31.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.001+0000 7f815bfff640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f816006c920 0x7f816006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.001+0000 7f815bfff640 1 -- 172.21.15.71:0/4076368410 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f816808b350 con 0x7f81740c99d0 2024-04-16T15:40:31.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.001+0000 7f81737fe640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f816006c920 0x7f816006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.002+0000 7f81737fe640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f816006c920 0x7f816006ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8164009a10 tx=0x7f8164008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.006+0000 7f815bfff640 1 -- 172.21.15.71:0/4076368410 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8168059440 con 0x7f81740c99d0 2024-04-16T15:40:31.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.183+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f817406e4c0 con 0x7f81740c99d0 2024-04-16T15:40:31.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.184+0000 7f815bfff640 1 -- 172.21.15.71:0/4076368410 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f8168058d40 con 0x7f81740c99d0 2024-04-16T15:40:31.183 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:31.183 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f816006c920 msgr2=0x7f816006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f816006c920 0x7f816006ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8164009a10 tx=0x7f8164008040 comp rx=0 tx=0).stop 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c99d0 msgr2=0x7f8174158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c99d0 0x7f8174158a70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f816800c470 tx=0x7f816800c930 comp rx=0 tx=0).stop 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 shutdown_connections 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f816006c920 0x7f816006ede0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8174153450 0x7f8174157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81740c99d0 0x7f8174158a70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 --2- 172.21.15.71:0/4076368410 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f81740c94c0 0x7f81740ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.186+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 >> 172.21.15.71:0/4076368410 conn(0x7f81740ad2f0 msgr2=0x7f81740ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.187+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 shutdown_connections 2024-04-16T15:40:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.187+0000 7f8179e0d640 1 -- 172.21.15.71:0/4076368410 wait complete. 2024-04-16T15:40:31.599 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-16T15:40:23.878007+0000', 'modified': '2024-04-16T15:40:28.915544+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6835/153100987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6833', 'nonce': 153100987}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 153100987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4296, 'qdb_cluster': [4296]} 2024-04-16T15:40:31.599 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-16T15:40:31.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T15:40:31.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.705+0000 7fb569d8d640 1 Processor -- start 2024-04-16T15:40:31.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.705+0000 7fb569d8d640 1 -- start start 2024-04-16T15:40:31.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb569d8d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb56406f8a0 0x7fb56406da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb569d8d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5640aab10 con 0x7fb5640c1b90 2024-04-16T15:40:31.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb569d8d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb56406fd30 con 0x7fb56414c6f0 2024-04-16T15:40:31.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb569d8d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb56414cad0 con 0x7fb56406f8a0 2024-04-16T15:40:31.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb562ffd640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fb5640c1b90 0x7fb5641599d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56042/0 (socket says 172.21.15.71:56042) 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb562ffd640 1 -- 172.21.15.71:0/106934760 learned_addr learned my addr 172.21.15.71:0/106934760 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb563fff640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb56406f8a0 0x7fb56406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538444981 0 0) 0x7fb5640aab10 con 0x7fb5640c1b90 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb54c003520 con 0x7fb5640c1b90 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1063997681 0 0) 0x7fb54c003520 con 0x7fb5640c1b90 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb56406f8a0 msgr2=0x7fb56406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb56406f8a0 0x7fb56406da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 >> v1:172.21.15.122:6789/0 conn(0x7fb56414c6f0 legacy=0x7fb5641565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5640aad80 con 0x7fb5640c1b90 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fb558001670 con 0x7fb5640c1b90 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 >> v1:172.21.15.71:6789/0 conn(0x7fb5640c1b90 legacy=0x7fb5641599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb56406f8a0 0x7fb54c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb54c003f20 0x7fb54c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c004d40 0x7fb54c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c00baa0 con 0x7fb54c003f20 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c00bc50 con 0x7fb54c004d40 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.706+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c00be50 con 0x7fb56406f8a0 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb562ffd640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb54c003f20 0x7fb54c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb5637fe640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb56406f8a0 0x7fb54c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb563fff640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c004d40 0x7fb54c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb562ffd640 1 -- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb56406f8a0 msgr2=0x7fb54c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb562ffd640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb56406f8a0 0x7fb54c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb562ffd640 1 -- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c004d40 msgr2=0x7fb54c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb562ffd640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c004d40 0x7fb54c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb562ffd640 1 -- 172.21.15.71:0/106934760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c003520 con 0x7fb54c003f20 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.707+0000 7fb562ffd640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb54c003f20 0x7fb54c004800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb55800c290 tx=0x7fb54c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8022fa6cf42420 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558007d50 con 0x7fb54c003f20 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb558030070 con 0x7fb54c003f20 2024-04-16T15:40:31.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb5627fc640 1 -- 172.21.15.71:0/106934760 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55802cda0 con 0x7fb54c003f20 2024-04-16T15:40:31.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb569d8d640 1 -- 172.21.15.71:0/106934760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb54c003f20 msgr2=0x7fb54c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb569d8d640 1 --2- 172.21.15.71:0/106934760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb54c003f20 0x7fb54c004800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb55800c290 tx=0x7fb54c005830 comp rx=0 tx=0).stop 2024-04-16T15:40:31.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb563fff640 1 -- 172.21.15.71:0/106934760 reap_dead start 2024-04-16T15:40:31.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb569d8d640 1 -- 172.21.15.71:0/106934760 shutdown_connections 2024-04-16T15:40:31.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb569d8d640 1 -- 172.21.15.71:0/106934760 >> 172.21.15.71:0/106934760 conn(0x7fb5640729d0 msgr2=0x7fb564072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb569d8d640 1 -- 172.21.15.71:0/106934760 shutdown_connections 2024-04-16T15:40:31.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.708+0000 7fb569d8d640 1 -- 172.21.15.71:0/106934760 wait complete. 2024-04-16T15:40:31.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.709+0000 7fb569d8d640 1 Processor -- start 2024-04-16T15:40:31.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb569d8d640 1 -- start start 2024-04-16T15:40:31.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb569d8d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb54c003f20 0x7fb564156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb569d8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5640ad030 0x7fb5641451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb569d8d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb564158a60 0x7fb5640ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb569d8d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb564159540 con 0x7fb564158a60 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb569d8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb56415a0e0 con 0x7fb5640ad030 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb569d8d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb5641591d0 con 0x7fb54c003f20 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb563fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb564158a60 0x7fb5640ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb563fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb564158a60 0x7fb5640ab2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49516/0 (socket says 172.21.15.71:49516) 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb563fff640 1 -- 172.21.15.71:0/731993357 learned_addr learned my addr 172.21.15.71:0/731993357 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb5637fe640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb54c003f20 0x7fb564156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb562ffd640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5640ad030 0x7fb5641451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.710+0000 7fb563fff640 1 -- 172.21.15.71:0/731993357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb54c003f20 msgr2=0x7fb564156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb563fff640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb54c003f20 0x7fb564156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb563fff640 1 -- 172.21.15.71:0/731993357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5640ad030 msgr2=0x7fb5641451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb563fff640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5640ad030 0x7fb5641451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb563fff640 1 -- 172.21.15.71:0/731993357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56414ffc0 con 0x7fb564158a60 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb563fff640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb564158a60 0x7fb5640ab2a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb564071580 tx=0x7fb55c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb54bfff640 1 -- 172.21.15.71:0/731993357 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c019020 con 0x7fb564158a60 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb54bfff640 1 -- 172.21.15.71:0/731993357 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c004240 con 0x7fb564158a60 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb569d8d640 1 -- 172.21.15.71:0/731993357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb564157b90 con 0x7fb564158a60 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb54bfff640 1 -- 172.21.15.71:0/731993357 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c00f740 con 0x7fb564158a60 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.711+0000 7fb569d8d640 1 -- 172.21.15.71:0/731993357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5640c2da0 con 0x7fb564158a60 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.712+0000 7fb54bfff640 1 -- 172.21.15.71:0/731993357 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb55c01c020 con 0x7fb564158a60 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.713+0000 7fb54bfff640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb55006c8f0 0x7fb55006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.713+0000 7fb5637fe640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb55006c8f0 0x7fb55006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.713+0000 7fb54bfff640 1 -- 172.21.15.71:0/731993357 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb55c08c190 con 0x7fb564158a60 2024-04-16T15:40:31.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.713+0000 7fb5637fe640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb55006c8f0 0x7fb55006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb554003fd0 tx=0x7fb554009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.714+0000 7fb56c972740 1 Processor -- start 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.714+0000 7fb56c972740 1 -- start start 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb56c972740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf122f6590 0x55bf12330e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb56c972740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf123313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb56c972740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf123323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb56c972740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bf1225a9c0 con 0x55bf122f6590 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb56c972740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bf12201650 con 0x55bf122f6970 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb56c972740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55bf12256980 con 0x55bf122f7620 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf122f6590 0x55bf12330e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf122f6590 0x55bf12330e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49532/0 (socket says 172.21.15.71:49532) 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 -- 172.21.15.71:0/1917272824 learned_addr learned my addr 172.21.15.71:0/1917272824 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 -- 172.21.15.71:0/1917272824 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 msgr2=0x55bf123323f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 --2- 172.21.15.71:0/1917272824 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf123323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 -- 172.21.15.71:0/1917272824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 msgr2=0x55bf123313c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 --2- 172.21.15.71:0/1917272824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf123313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 -- 172.21.15.71:0/1917272824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf11f6d2d0 con 0x55bf122f6590 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5637fe640 1 --2- 172.21.15.71:0/1917272824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf122f6590 0x55bf12330e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5540021d0 tx=0x7fb554014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5101224301fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5497fa640 1 -- 172.21.15.71:0/1917272824 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554028070 con 0x55bf122f6590 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5497fa640 1 -- 172.21.15.71:0/1917272824 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5540143d0 con 0x55bf122f6590 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.715+0000 7fb5497fa640 1 -- 172.21.15.71:0/1917272824 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554014780 con 0x55bf122f6590 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 -- 172.21.15.71:0/1917272824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf122f6590 msgr2=0x55bf12330e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 --2- 172.21.15.71:0/1917272824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf122f6590 0x55bf12330e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5540021d0 tx=0x7fb554014000 comp rx=0 tx=0).stop 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 -- 172.21.15.71:0/1917272824 shutdown_connections 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 --2- 172.21.15.71:0/1917272824 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf123323f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 --2- 172.21.15.71:0/1917272824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf123313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 --2- 172.21.15.71:0/1917272824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf122f6590 0x55bf12330e80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 -- 172.21.15.71:0/1917272824 >> 172.21.15.71:0/1917272824 conn(0x55bf122f2c60 msgr2=0x55bf12317670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 -- 172.21.15.71:0/1917272824 shutdown_connections 2024-04-16T15:40:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.716+0000 7fb56c972740 1 -- 172.21.15.71:0/1917272824 wait complete. 2024-04-16T15:40:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.717+0000 7fb56c972740 1 -- start start 2024-04-16T15:40:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.717+0000 7fb56c972740 1 Processor -- start 2024-04-16T15:40:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf1231afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf12314690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12314bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bf123351b0 con 0x55bf1231b4e0 2024-04-16T15:40:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bf12214a10 con 0x55bf122f6970 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55bf12333d90 con 0x55bf122f7620 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 10 client.0 _ll_register_callbacks cb 0x55bf12214400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12314bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12314bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49544/0 (socket says 172.21.15.71:49544) 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 -- 172.21.15.71:0/3393499621 learned_addr learned my addr 172.21.15.71:0/3393499621 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb5637fe640 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf1231afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb562ffd640 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf12314690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 -- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 msgr2=0x55bf12314690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf12314690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 -- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 msgr2=0x55bf1231afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf1231afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 -- 172.21.15.71:0/3393499621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf120376b0 con 0x55bf1231b4e0 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb563fff640 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12314bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb55c001eb0 tx=0x7fb55c05a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e0eca60e1fbc30 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb5427fc640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb5427fc640 1 -- 172.21.15.71:0/3393499621 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c000e60 con 0x55bf1231b4e0 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb5427fc640 1 -- 172.21.15.71:0/3393499621 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb55c062360 con 0x55bf1231b4e0 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 1 -- 172.21.15.71:0/3393499621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5540143d0 con 0x55bf1231b4e0 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb5427fc640 1 -- 172.21.15.71:0/3393499621 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fb55c05dfc0 con 0x55bf1231b4e0 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 10 client.4431 fetch_fsmap learned FSMap version 7 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.718+0000 7fb56c972740 1 -- 172.21.15.71:0/3393499621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55bf123196b0 con 0x55bf1231b4e0 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb5427fc640 1 -- 172.21.15.71:0/3393499621 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5540143d0 con 0x55bf1231b4e0 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb5427fc640 1 -- 172.21.15.71:0/3393499621 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fb55c059060 con 0x55bf1231b4e0 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb56c972740 10 client.4431 fetch_fsmap finished waiting for FSMap version 7 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb56c972740 10 client.4431 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb56c972740 20 client.4431 populate_metadata read hostname 'smithi071' 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb56c972740 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x55bf12346d20 0x55bf12351bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb56c972740 4 client.4431 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T15:40:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb56c972740 1 -- 172.21.15.71:0/3393499621 --> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf122683e0 con 0x55bf12346d20 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb5637fe640 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x55bf12346d20 0x55bf12351bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb5427fc640 10 client.4431 ms_handle_connect on v2:172.21.15.122:6833/153100987 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.719+0000 7fb5637fe640 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x55bf12346d20 0x55bf12351bd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ae90efc6a1188d server_cookie=c152d10942eb5e64 in_seq=0 out_seq=0 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb5427fc640 1 -- 172.21.15.71:0/3393499621 <== mds.0 v2:172.21.15.122:6833/153100987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf122683e0 con 0x55bf12346d20 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb5427fc640 10 client.4431 handle_command_reply: tid=0 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 client.4431 shutdown 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 -- 172.21.15.71:0/3393499621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 msgr2=0x55bf12314bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12314bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb55c001eb0 tx=0x7fb55c05a310 comp rx=0 tx=0).stop 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 -- 172.21.15.71:0/3393499621 shutdown_connections 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12314bd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x55bf12346d20 0x55bf12351bd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf12314690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb5427fc640 0 client.4431 ms_handle_reset on v2:172.21.15.122:6833/153100987 2024-04-16T15:40:31.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 --2- 172.21.15.71:0/3393499621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf1231afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 -- 172.21.15.71:0/3393499621 >> 172.21.15.71:0/3393499621 conn(0x55bf122f6590 msgr2=0x55bf12317b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 -- 172.21.15.71:0/3393499621 shutdown_connections 2024-04-16T15:40:31.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 1 -- 172.21.15.71:0/3393499621 wait complete. 2024-04-16T15:40:31.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.720+0000 7fb56c972740 20 client.4431 trim_cache size 0 max 16384 2024-04-16T15:40:31.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.738+0000 7fb56c972740 1 Processor -- start 2024-04-16T15:40:31.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.738+0000 7fb56c972740 1 -- start start 2024-04-16T15:40:31.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb56c972740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf12318430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb56c972740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf12318970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb56c972740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12335a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb56c972740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641f8c30 con 0x55bf1231b4e0 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb56c972740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641f8100 con 0x55bf122f6970 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb56c972740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb5641f85e0 con 0x55bf122f7620 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb563fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12335a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb563fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12335a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49546/0 (socket says 172.21.15.71:49546) 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb563fff640 1 -- 172.21.15.71:0/488793443 learned_addr learned my addr 172.21.15.71:0/488793443 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb5637fe640 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf12318430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.739+0000 7fb562ffd640 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf12318970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb563fff640 1 -- 172.21.15.71:0/488793443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 msgr2=0x55bf12318970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb563fff640 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf12318970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb563fff640 1 -- 172.21.15.71:0/488793443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 msgr2=0x55bf12318430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb563fff640 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf12318430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb563fff640 1 -- 172.21.15.71:0/488793443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf12333250 con 0x55bf1231b4e0 2024-04-16T15:40:31.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb563fff640 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12335a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb55c059ba0 tx=0x7fb55c08b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd2368b3cae9c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb548ff9640 1 -- 172.21.15.71:0/488793443 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c000e60 con 0x55bf1231b4e0 2024-04-16T15:40:31.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb548ff9640 1 -- 172.21.15.71:0/488793443 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c072100 con 0x55bf1231b4e0 2024-04-16T15:40:31.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb548ff9640 1 -- 172.21.15.71:0/488793443 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c062360 con 0x55bf1231b4e0 2024-04-16T15:40:31.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb56c972740 1 -- 172.21.15.71:0/488793443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 msgr2=0x55bf12335a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.740+0000 7fb56c972740 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12335a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb55c059ba0 tx=0x7fb55c08b000 comp rx=0 tx=0).stop 2024-04-16T15:40:31.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 -- 172.21.15.71:0/488793443 shutdown_connections 2024-04-16T15:40:31.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12335a80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f7620 0x55bf12318970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 --2- 172.21.15.71:0/488793443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f6970 0x55bf12318430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 -- 172.21.15.71:0/488793443 >> 172.21.15.71:0/488793443 conn(0x55bf122f6590 msgr2=0x55bf12315370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 -- 172.21.15.71:0/488793443 shutdown_connections 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 -- 172.21.15.71:0/488793443 wait complete. 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 -- start start 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.741+0000 7fb56c972740 1 Processor -- start 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb56c972740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f6970 0x55bf12356590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb56c972740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f7620 0x55bf123ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb56c972740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12318430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb56c972740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bf1232fbd0 con 0x55bf1231b4e0 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb56c972740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bf123351b0 con 0x55bf122f7620 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb56c972740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55bf12343ad0 con 0x55bf122f6970 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb56c972740 10 client.0 _ll_register_callbacks cb 0x7fb55c011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12318430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12318430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49548/0 (socket says 172.21.15.71:49548) 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 -- 172.21.15.71:0/347818096 learned_addr learned my addr 172.21.15.71:0/347818096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb5637fe640 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f6970 0x55bf12356590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb562ffd640 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f7620 0x55bf123ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 -- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f6970 msgr2=0x55bf12356590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f6970 0x55bf12356590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 -- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f7620 msgr2=0x55bf123ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f7620 0x55bf123ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 -- 172.21.15.71:0/347818096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf123b1b10 con 0x55bf1231b4e0 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb562ffd640 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f7620 0x55bf123ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb563fff640 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12318430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb55c05a4c0 tx=0x7fb55c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179bdf9f967d9c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.742+0000 7fb5427fc640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb5427fc640 1 -- 172.21.15.71:0/347818096 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c000e60 con 0x55bf1231b4e0 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb5427fc640 1 -- 172.21.15.71:0/347818096 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb55c062360 con 0x55bf1231b4e0 2024-04-16T15:40:31.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 1 -- 172.21.15.71:0/347818096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb55c072100 con 0x55bf1231b4e0 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb5427fc640 1 -- 172.21.15.71:0/347818096 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fb55c0625a0 con 0x55bf1231b4e0 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 10 client.4437 fetch_fsmap learned FSMap version 7 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 1 -- 172.21.15.71:0/347818096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55bf12342f20 con 0x55bf1231b4e0 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb5427fc640 1 -- 172.21.15.71:0/347818096 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c072100 con 0x55bf1231b4e0 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb5427fc640 1 -- 172.21.15.71:0/347818096 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fb55c0601d0 con 0x55bf1231b4e0 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 10 client.4437 fetch_fsmap finished waiting for FSMap version 7 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 10 client.4437 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 20 client.4437 populate_metadata read hostname 'smithi071' 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x55bf12346d20 0x55bf123189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 4 client.4437 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb56c972740 1 -- 172.21.15.71:0/347818096 --> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] -- command(tid 0: {"prefix": "status"}) -- 0x55bf122f33c0 con 0x55bf12346d20 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.743+0000 7fb5637fe640 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x55bf12346d20 0x55bf123189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb5427fc640 10 client.4437 ms_handle_connect on v2:172.21.15.122:6833/153100987 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb5637fe640 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x55bf12346d20 0x55bf123189e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f7c4451b0eb116 server_cookie=c22c4102c61269e2 in_seq=0 out_seq=0 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb5427fc640 1 -- 172.21.15.71:0/347818096 <== mds.0 v2:172.21.15.122:6833/153100987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55bf122f33c0 con 0x55bf12346d20 2024-04-16T15:40:31.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb5427fc640 10 client.4437 handle_command_reply: tid=0 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 client.4437 shutdown 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 -- 172.21.15.71:0/347818096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 msgr2=0x55bf12318430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12318430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb55c05a4c0 tx=0x7fb55c05c040 comp rx=0 tx=0).stop 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 -- 172.21.15.71:0/347818096 shutdown_connections 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf1231b4e0 0x55bf12318430 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf122f7620 0x55bf123ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x55bf12346d20 0x55bf123189e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 --2- 172.21.15.71:0/347818096 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55bf122f6970 0x55bf12356590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb5427fc640 0 client.4437 ms_handle_reset on v2:172.21.15.122:6833/153100987 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 -- 172.21.15.71:0/347818096 >> 172.21.15.71:0/347818096 conn(0x55bf122f6590 msgr2=0x55bf1233ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 -- 172.21.15.71:0/347818096 shutdown_connections 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 1 -- 172.21.15.71:0/347818096 wait complete. 2024-04-16T15:40:31.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb56c972740 20 client.4437 trim_cache size 0 max 16384 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "c5697e20-5e32-4f13-b48e-2f7b8429c0b4", 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4296, 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 7.2229411040000002, 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 7, 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 24, 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 23, 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 14.388558134 2024-04-16T15:40:31.749 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:40:31.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb569d8d640 1 -- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb55006c8f0 msgr2=0x7fb55006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb569d8d640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb55006c8f0 0x7fb55006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb554003fd0 tx=0x7fb554009040 comp rx=0 tx=0).stop 2024-04-16T15:40:31.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb569d8d640 1 -- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb564158a60 msgr2=0x7fb5640ab2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.744+0000 7fb569d8d640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb564158a60 0x7fb5640ab2a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb564071580 tx=0x7fb55c011cf0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.745+0000 7fb569d8d640 1 -- 172.21.15.71:0/731993357 shutdown_connections 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.745+0000 7fb569d8d640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb55006c8f0 0x7fb55006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.745+0000 7fb569d8d640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb564158a60 0x7fb5640ab2a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.745+0000 7fb569d8d640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5640ad030 0x7fb5641451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.745+0000 7fb569d8d640 1 --2- 172.21.15.71:0/731993357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb54c003f20 0x7fb564156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.745+0000 7fb569d8d640 1 -- 172.21.15.71:0/731993357 >> 172.21.15.71:0/731993357 conn(0x7fb5640729d0 msgr2=0x7fb56406c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.745+0000 7fb569d8d640 1 -- 172.21.15.71:0/731993357 shutdown_connections 2024-04-16T15:40:31.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.745+0000 7fb569d8d640 1 -- 172.21.15.71:0/731993357 wait complete. 2024-04-16T15:40:31.754 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T15:40:31.754 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T15:40:31.754 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-16T15:40:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6f055640 1 Processor -- start 2024-04-16T15:40:31.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6f055640 1 -- start start 2024-04-16T15:40:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6f055640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68070810 0x7f3f6806c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6f055640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f3f6806ce80 0x7f3f68072520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6f055640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f680c78b0 con 0x7f3f68151010 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6f055640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f3f68073f80 con 0x7f3f6806ce80 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6f055640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3f680735e0 con 0x7f3f68070810 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6d5f2640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f3f6806ce80 0x7f3f68072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6cdf1640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f3f68151010 0x7f3f680c7030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56052/0 (socket says 172.21.15.71:56052) 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.859+0000 7f3f6cdf1640 1 -- 172.21.15.71:0/3542618325 learned_addr learned my addr 172.21.15.71:0/3542618325 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.860+0000 7f3f6d5f2640 1 -- 172.21.15.71:0/3542618325 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68070810 msgr2=0x7f3f6806c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.860+0000 7f3f67fff640 1 --2- 172.21.15.71:0/3542618325 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68070810 0x7f3f6806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.860+0000 7f3f6d5f2640 1 --2- 172.21.15.71:0/3542618325 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68070810 0x7f3f6806c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.860+0000 7f3f677fe640 1 -- 172.21.15.71:0/3542618325 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3103819033 0 0) 0x7f3f680c78b0 con 0x7f3f68151010 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.860+0000 7f3f6d5f2640 1 -- 172.21.15.71:0/3542618325 >> v1:172.21.15.71:6789/0 conn(0x7f3f68151010 legacy=0x7f3f680c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.860+0000 7f3f6d5f2640 1 -- 172.21.15.71:0/3542618325 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f680738c0 con 0x7f3f6806ce80 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.860+0000 7f3f67fff640 1 --2- 172.21.15.71:0/3542618325 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68070810 0x7f3f6806c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.860+0000 7f3f6d5f2640 1 --2- 172.21.15.71:0/3542618325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f68072520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009d90 tx=0x7f3f6002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ebe536627ea08f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f677fe640 1 -- 172.21.15.71:0/3542618325 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60039070 con 0x7f3f6806ce80 2024-04-16T15:40:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f677fe640 1 -- 172.21.15.71:0/3542618325 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6002cbc0 con 0x7f3f6806ce80 2024-04-16T15:40:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f677fe640 1 -- 172.21.15.71:0/3542618325 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60034440 con 0x7f3f6806ce80 2024-04-16T15:40:31.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f6f055640 1 -- 172.21.15.71:0/3542618325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 msgr2=0x7f3f68072520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f6f055640 1 --2- 172.21.15.71:0/3542618325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f68072520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009d90 tx=0x7f3f6002be10 comp rx=0 tx=0).stop 2024-04-16T15:40:31.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f6f055640 1 -- 172.21.15.71:0/3542618325 shutdown_connections 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f6f055640 1 --2- 172.21.15.71:0/3542618325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f68072520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f6f055640 1 --2- 172.21.15.71:0/3542618325 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68070810 0x7f3f6806c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f6f055640 1 -- 172.21.15.71:0/3542618325 >> 172.21.15.71:0/3542618325 conn(0x7f3f680c45d0 msgr2=0x7f3f680c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f6f055640 1 -- 172.21.15.71:0/3542618325 shutdown_connections 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.861+0000 7f3f6f055640 1 -- 172.21.15.71:0/3542618325 wait complete. 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.862+0000 7f3f6f055640 1 Processor -- start 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.862+0000 7f3f6f055640 1 -- start start 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6f055640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f6806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6f055640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f68070810 0x7f3f68149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6f055640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68151010 0x7f3f6814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6f055640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68151570 con 0x7f3f6806ce80 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6f055640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f681513f0 con 0x7f3f68070810 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6f055640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3f68072a60 con 0x7f3f68151010 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6cdf1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f6806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6cdf1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f6806f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49576/0 (socket says 172.21.15.71:49576) 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.863+0000 7f3f6cdf1640 1 -- 172.21.15.71:0/2629524367 learned_addr learned my addr 172.21.15.71:0/2629524367 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f67fff640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f68070810 0x7f3f68149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6d5f2640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68151010 0x7f3f6814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6cdf1640 1 -- 172.21.15.71:0/2629524367 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68151010 msgr2=0x7f3f6814a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6cdf1640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68151010 0x7f3f6814a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6cdf1640 1 -- 172.21.15.71:0/2629524367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f68070810 msgr2=0x7f3f68149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6cdf1640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f68070810 0x7f3f68149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6cdf1640 1 -- 172.21.15.71:0/2629524367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f681e0c00 con 0x7f3f6806ce80 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6cdf1640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f6806f740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00c430 tx=0x7f3f5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f657fa640 1 -- 172.21.15.71:0/2629524367 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0074e0 con 0x7f3f6806ce80 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f657fa640 1 -- 172.21.15.71:0/2629524367 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5c0047e0 con 0x7f3f6806ce80 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f681e0ec0 con 0x7f3f6806ce80 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f657fa640 1 -- 172.21.15.71:0/2629524367 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c004bd0 con 0x7f3f6806ce80 2024-04-16T15:40:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.864+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f68154580 con 0x7f3f6806ce80 2024-04-16T15:40:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.865+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f68151e40 con 0x7f3f6806ce80 2024-04-16T15:40:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.865+0000 7f3f657fa640 1 -- 172.21.15.71:0/2629524367 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f5c011070 con 0x7f3f6806ce80 2024-04-16T15:40:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.866+0000 7f3f657fa640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3f4c06c8f0 0x7f3f4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.866+0000 7f3f67fff640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3f4c06c8f0 0x7f3f4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.866+0000 7f3f657fa640 1 -- 172.21.15.71:0/2629524367 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3f5c08ba20 con 0x7f3f6806ce80 2024-04-16T15:40:31.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.866+0000 7f3f67fff640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3f4c06c8f0 0x7f3f4c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f680c3740 tx=0x7f3f54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:31.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:31.870+0000 7f3f657fa640 1 -- 172.21.15.71:0/2629524367 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f68151e40 con 0x7f3f6806ce80 2024-04-16T15:40:32.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.083+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f6806d470 con 0x7f3f6806ce80 2024-04-16T15:40:32.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.083+0000 7f3f657fa640 1 -- 172.21.15.71:0/2629524367 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f3f6806d470 con 0x7f3f6806ce80 2024-04-16T15:40:32.082 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T15:40:32.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3f4c06c8f0 msgr2=0x7f3f4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:32.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3f4c06c8f0 0x7f3f4c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f680c3740 tx=0x7f3f54008040 comp rx=0 tx=0).stop 2024-04-16T15:40:32.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 msgr2=0x7f3f6806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f6806f740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00c430 tx=0x7f3f5c00c8f0 comp rx=0 tx=0).stop 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 shutdown_connections 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3f4c06c8f0 0x7f3f4c06edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f68151010 0x7f3f6814a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f68070810 0x7f3f68149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 --2- 172.21.15.71:0/2629524367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6806ce80 0x7f3f6806f740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.085+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 >> 172.21.15.71:0/2629524367 conn(0x7f3f680c45d0 msgr2=0x7f3f6814e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.086+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 shutdown_connections 2024-04-16T15:40:32.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32.086+0000 7f3f6f055640 1 -- 172.21.15.71:0/2629524367 wait complete. 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T15:40:32.095 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T15:40:32.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:32.096 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-04-16T15:40:32.111 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T15:40:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T15:40:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-16T15:40:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42753sec preferred_lft 42753sec 2024-04-16T15:40:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2024-04-16T15:40:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:32.113 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T15:40:32.113 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:32.113 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:32.113 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2024-04-16T15:40:32.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2024-04-16T15:40:32.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2024-04-16T15:40:32.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T15:40:32.114 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:32.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:32.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:32.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:32.353 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T15:40:32.382 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-04-16T15:40:32.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:32.384 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-04-16T15:40:32.457 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-04-16T15:40:32.457 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T15:40:32.457 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-16T15:40:32.457 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-16T15:40:32.457 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T15:40:32.457 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:32.458 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:32.458 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-16T15:40:32.458 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-16T15:40:32.458 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-16T15:40:32.458 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:32.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:32.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:32.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:32.755 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-04-16T15:40:32.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:32.771 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-04-16T15:40:32.826 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:32.827 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:32.827 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T15:40:32.827 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-16T15:40:32.827 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:32.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:32.952 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-16T15:40:32.952 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:32.952 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:32.952 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-16T15:40:32.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T15:40:32.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-16T15:40:32.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-16T15:40:32.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-16T15:40:32.953 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:33.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:33.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:33.121 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:33.121 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:33.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-04-16T15:40:33.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T15:40:33.121 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:33.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:33.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:33.205 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-16T15:40:33.205 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T15:40:33.205 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-16T15:40:33.221 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T15:40:33.221 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T15:40:33.221 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-16T15:40:33.278 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-04-16T15:40:33.347 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2024-04-16T15:40:33.404 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/selinux 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2024-04-16T15:40:33.405 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2024-04-16T15:40:33.406 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2024-04-16T15:40:33.406 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2024-04-16T15:40:33.406 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2024-04-16T15:40:33.406 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T15:40:33.406 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T15:40:33.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:33.406 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-04-16T15:40:33.460 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-16T15:40:33.460 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-16T15:40:33.503 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-04-16T15:40:33.531 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T15:40:33.588 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2024-04-16T15:40:33.588 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/selinux 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T15:40:33.589 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2024-04-16T15:40:33.590 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2024-04-16T15:40:33.591 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2024-04-16T15:40:33.591 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2024-04-16T15:40:33.591 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2024-04-16T15:40:33.591 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2024-04-16T15:40:33.591 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T15:40:33.591 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T15:40:33.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:33.591 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-04-16T15:40:33.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:2024-04-16T15:40:33.594+0000 7fc48dc41480 -1 init, newargv = 0x5590e7b33ff0 newargc=15 2024-04-16T15:40:33.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[45297]: starting ceph client 2024-04-16T15:40:33.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[45297]: starting fuse 2024-04-16T15:40:33.608 INFO:teuthology.orchestra.run.smithi071.stdout:41 2024-04-16T15:40:33.608 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-16T15:40:33.608 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2024-04-16T15:40:33.608 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2024-04-16T15:40:33.608 DEBUG:teuthology.orchestra.run.smithi071:> import re 2024-04-16T15:40:33.608 DEBUG:teuthology.orchestra.run.smithi071:> import os 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> 2024-04-16T15:40:33.609 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> return f 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.admin")) 2024-04-16T15:40:33.610 DEBUG:teuthology.orchestra.run.smithi071:> ' 2024-04-16T15:40:33.731 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.admin.45297.asok 2024-04-16T15:40:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:33.744 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.45297.asok 2024-04-16T15:40:33.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:33.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.45297.asok status 2024-04-16T15:40:33.882 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f", 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev)", 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "admin", 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "45297", 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4452, 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2024-04-16T15:40:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 4452 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3683915281 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3683915281 2024-04-16T15:40:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.4452 192.168.0.1:0/3683915281", 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/3683915281", 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 7, 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 24, 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2024-04-16T15:40:33.885 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:40:33.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:33.891 DEBUG:teuthology.orchestra.run.smithi071:> stat --file-system '--printf=%T 2024-04-16T15:40:33.891 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-16T15:40:33.907 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2024-04-16T15:40:33.907 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-16T15:40:33.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:33.908 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-16T15:40:33.977 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-16T15:40:34.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T15:40:34.090 INFO:teuthology.orchestra.run.smithi071.stdout:# file: . 2024-04-16T15:40:34.091 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-16T15:40:34.091 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:34.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:34.097 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:40:34.097 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:40:34.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:34.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.258+0000 7fc588ff4640 1 Processor -- start 2024-04-16T15:40:34.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.258+0000 7fc588ff4640 1 -- start start 2024-04-16T15:40:34.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc588ff4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5840c4350 0x7fc5840c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc588ff4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc584159740 0x7fc584159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc588ff4640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5840c82d0 con 0x7fc584153570 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc588ff4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc5840c41a0 con 0x7fc584159740 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc588ff4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc584155d70 con 0x7fc5840c4350 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58259c640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fc584153570 0x7fc5840c5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56064/0 (socket says 172.21.15.71:56064) 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58259c640 1 -- 172.21.15.71:0/861351607 learned_addr learned my addr 172.21.15.71:0/861351607 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 910906466 0 0) 0x7fc5840c82d0 con 0x7fc584153570 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc581d9b640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5840c4350 0x7fc5840c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc582d9d640 1 --2- 172.21.15.71:0/861351607 >> v2:172.21.15.122:3300/0 conn(0x7fc584159740 0x7fc584159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc570003520 con 0x7fc584153570 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2131431582 0 0) 0x7fc570003520 con 0x7fc584153570 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5840c4350 msgr2=0x7fc5840c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5840c4350 0x7fc5840c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 >> v2:172.21.15.122:3300/0 conn(0x7fc584159740 msgr2=0x7fc584159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 --2- 172.21.15.71:0/861351607 >> v2:172.21.15.122:3300/0 conn(0x7fc584159740 0x7fc584159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5840c8d50 con 0x7fc584153570 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fc574001670 con 0x7fc584153570 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 >> v1:172.21.15.71:6789/0 conn(0x7fc584153570 legacy=0x7fc5840c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5700039b0 0x7fc570003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc570004800 0x7fc570004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5840c4350 0x7fc5700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc584155d70 con 0x7fc570004800 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc57000b9c0 con 0x7fc5700039b0 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.259+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc57000bbc0 con 0x7fc5840c4350 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.260+0000 7fc58259c640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5700039b0 0x7fc570003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.260+0000 7fc581d9b640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc570004800 0x7fc570004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.260+0000 7fc582d9d640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5840c4350 0x7fc5700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.260+0000 7fc581d9b640 1 -- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5840c4350 msgr2=0x7fc5700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.260+0000 7fc581d9b640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5840c4350 0x7fc5700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.260+0000 7fc581d9b640 1 -- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5700039b0 msgr2=0x7fc570003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.260+0000 7fc581d9b640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5700039b0 0x7fc570003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.260+0000 7fc581d9b640 1 -- 172.21.15.71:0/861351607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc570003520 con 0x7fc570004800 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc581d9b640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc570004800 0x7fc570004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009ac0 tx=0x7fc570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef536c32debb6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c039070 con 0x7fc570004800 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c02c6b0 con 0x7fc570004800 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc58159a640 1 -- 172.21.15.71:0/861351607 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c009690 con 0x7fc570004800 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc588ff4640 1 -- 172.21.15.71:0/861351607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc570004800 msgr2=0x7fc570004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc588ff4640 1 --2- 172.21.15.71:0/861351607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc570004800 0x7fc570004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009ac0 tx=0x7fc570005830 comp rx=0 tx=0).stop 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc582d9d640 1 -- 172.21.15.71:0/861351607 reap_dead start 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc588ff4640 1 -- 172.21.15.71:0/861351607 shutdown_connections 2024-04-16T15:40:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc588ff4640 1 -- 172.21.15.71:0/861351607 >> 172.21.15.71:0/861351607 conn(0x7fc5840c96e0 msgr2=0x7fc5840c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:34.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc588ff4640 1 -- 172.21.15.71:0/861351607 shutdown_connections 2024-04-16T15:40:34.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.261+0000 7fc588ff4640 1 -- 172.21.15.71:0/861351607 wait complete. 2024-04-16T15:40:34.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc588ff4640 1 Processor -- start 2024-04-16T15:40:34.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc588ff4640 1 -- start start 2024-04-16T15:40:34.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc588ff4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc570004800 0x7fc5841589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc588ff4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5841eec70 0x7fc5841ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc588ff4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5841ef630 0x7fc5840a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc588ff4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc58415a110 con 0x7fc5841ef630 2024-04-16T15:40:34.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc588ff4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5840c8ff0 con 0x7fc570004800 2024-04-16T15:40:34.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc588ff4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc58415a290 con 0x7fc5841eec70 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc582d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5841ef630 0x7fc5840a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc582d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5841ef630 0x7fc5840a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49590/0 (socket says 172.21.15.71:49590) 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.263+0000 7fc582d9d640 1 -- 172.21.15.71:0/3763289063 learned_addr learned my addr 172.21.15.71:0/3763289063 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc581d9b640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5841eec70 0x7fc5841ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc58259c640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc570004800 0x7fc5841589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc582d9d640 1 -- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5841eec70 msgr2=0x7fc5841ef0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc582d9d640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5841eec70 0x7fc5841ef0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc582d9d640 1 -- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc570004800 msgr2=0x7fc5841589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc582d9d640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc570004800 0x7fc5841589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc582d9d640 1 -- 172.21.15.71:0/3763289063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58414b400 con 0x7fc5841ef630 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc58259c640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc570004800 0x7fc5841589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc582d9d640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5841ef630 0x7fc5840a9220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc5840a8f40 tx=0x7fc57c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc57affd640 1 -- 172.21.15.71:0/3763289063 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00fd80 con 0x7fc5841ef630 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc57affd640 1 -- 172.21.15.71:0/3763289063 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c015430 con 0x7fc5841ef630 2024-04-16T15:40:34.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc57affd640 1 -- 172.21.15.71:0/3763289063 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c015820 con 0x7fc5841ef630 2024-04-16T15:40:34.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5840abcd0 con 0x7fc5841ef630 2024-04-16T15:40:34.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.264+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5840aafa0 con 0x7fc5841ef630 2024-04-16T15:40:34.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.265+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc584068d20 con 0x7fc5841ef630 2024-04-16T15:40:34.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.265+0000 7fc57affd640 1 -- 172.21.15.71:0/3763289063 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc57c0159c0 con 0x7fc5841ef630 2024-04-16T15:40:34.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.266+0000 7fc57affd640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc56406c920 0x7fc56406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.266+0000 7fc58259c640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc56406c920 0x7fc56406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.266+0000 7fc57affd640 1 -- 172.21.15.71:0/3763289063 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc57c08cbf0 con 0x7fc5841ef630 2024-04-16T15:40:34.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.266+0000 7fc58259c640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc56406c920 0x7fc56406ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc574007fd0 tx=0x7fc574009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.270+0000 7fc57affd640 1 -- 172.21.15.71:0/3763289063 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc57c019030 con 0x7fc5841ef630 2024-04-16T15:40:34.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.451+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc584159240 con 0x7fc5841ef630 2024-04-16T15:40:34.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.452+0000 7fc57affd640 1 -- 172.21.15.71:0/3763289063 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7fc57c010040 con 0x7fc5841ef630 2024-04-16T15:40:34.451 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:34.451 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc56406c920 msgr2=0x7fc56406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc56406c920 0x7fc56406ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc574007fd0 tx=0x7fc574009040 comp rx=0 tx=0).stop 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5841ef630 msgr2=0x7fc5840a9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5841ef630 0x7fc5840a9220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc5840a8f40 tx=0x7fc57c00f580 comp rx=0 tx=0).stop 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 shutdown_connections 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc56406c920 0x7fc56406ede0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5841ef630 0x7fc5840a9220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5841eec70 0x7fc5841ef0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 --2- 172.21.15.71:0/3763289063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc570004800 0x7fc5841589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 >> 172.21.15.71:0/3763289063 conn(0x7fc5840c96e0 msgr2=0x7fc584154380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:34.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 shutdown_connections 2024-04-16T15:40:34.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.454+0000 7fc588ff4640 1 -- 172.21.15.71:0/3763289063 wait complete. 2024-04-16T15:40:34.464 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:40:34.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:34.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.571+0000 7f7e166c8640 1 Processor -- start 2024-04-16T15:40:34.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.571+0000 7f7e166c8640 1 -- start start 2024-04-16T15:40:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e166c8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f7e100ac5c0 0x7f7e10154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e166c8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7e1006e0a0 0x7f7e100ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e166c8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e10158110 con 0x7f7e10157c80 2024-04-16T15:40:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e166c8640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f7e101543a0 con 0x7f7e100ac5c0 2024-04-16T15:40:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e166c8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7e100c3e20 con 0x7f7e1006e0a0 2024-04-16T15:40:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e0ffff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f7e100ac5c0 0x7f7e10154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e14c65640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f7e10157c80 0x7f7e100c3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56072/0 (socket says 172.21.15.71:56072) 2024-04-16T15:40:34.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e14c65640 1 -- 172.21.15.71:0/993659713 learned_addr learned my addr 172.21.15.71:0/993659713 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e0f7fe640 1 --2- 172.21.15.71:0/993659713 >> v2:172.21.15.122:3300/0 conn(0x7f7e1006e0a0 0x7f7e100ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e0ffff640 1 -- 172.21.15.71:0/993659713 >> v2:172.21.15.122:3300/0 conn(0x7f7e1006e0a0 msgr2=0x7f7e100ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e0ffff640 1 --2- 172.21.15.71:0/993659713 >> v2:172.21.15.122:3300/0 conn(0x7f7e1006e0a0 0x7f7e100ac9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e0ffff640 1 -- 172.21.15.71:0/993659713 >> v1:172.21.15.71:6789/0 conn(0x7f7e10157c80 legacy=0x7f7e100c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e0ffff640 1 -- 172.21.15.71:0/993659713 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e100a9e90 con 0x7f7e100ac5c0 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.572+0000 7f7e0effd640 1 -- 172.21.15.71:0/993659713 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1340204947 0 0) 0x7f7e10158110 con 0x7f7e10157c80 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.573+0000 7f7e0ffff640 1 --2- 172.21.15.71:0/993659713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e10154830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7e100c94f0 tx=0x7f7e0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f343bce5fb5af81 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.573+0000 7f7e0effd640 1 -- 172.21.15.71:0/993659713 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00035020 con 0x7f7e100ac5c0 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.573+0000 7f7e0effd640 1 -- 172.21.15.71:0/993659713 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e00031830 con 0x7f7e100ac5c0 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.573+0000 7f7e0effd640 1 -- 172.21.15.71:0/993659713 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00031c00 con 0x7f7e100ac5c0 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.574+0000 7f7e166c8640 1 -- 172.21.15.71:0/993659713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 msgr2=0x7f7e10154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.574+0000 7f7e166c8640 1 --2- 172.21.15.71:0/993659713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e10154830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7e100c94f0 tx=0x7f7e0002b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.574+0000 7f7e166c8640 1 -- 172.21.15.71:0/993659713 shutdown_connections 2024-04-16T15:40:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.574+0000 7f7e166c8640 1 --2- 172.21.15.71:0/993659713 >> v2:172.21.15.122:3300/0 conn(0x7f7e1006e0a0 0x7f7e100ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.574+0000 7f7e166c8640 1 --2- 172.21.15.71:0/993659713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e10154830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.574+0000 7f7e166c8640 1 -- 172.21.15.71:0/993659713 >> 172.21.15.71:0/993659713 conn(0x7f7e100c8470 msgr2=0x7f7e100c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.574+0000 7f7e166c8640 1 -- 172.21.15.71:0/993659713 shutdown_connections 2024-04-16T15:40:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.574+0000 7f7e166c8640 1 -- 172.21.15.71:0/993659713 wait complete. 2024-04-16T15:40:34.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.575+0000 7f7e166c8640 1 Processor -- start 2024-04-16T15:40:34.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.575+0000 7f7e166c8640 1 -- start start 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e166c8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1006e0a0 0x7f7e10156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e166c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e101e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e166c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e10157c80 0x7f7e101ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e166c8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10154030 con 0x7f7e100ac5c0 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e166c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e100aa450 con 0x7f7e10157c80 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e166c8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7e10153d00 con 0x7f7e1006e0a0 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e101e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e101e9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49596/0 (socket says 172.21.15.71:49596) 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 -- 172.21.15.71:0/1703258085 learned_addr learned my addr 172.21.15.71:0/1703258085 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0ffff640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1006e0a0 0x7f7e10156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e14c65640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e10157c80 0x7f7e101ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 -- 172.21.15.71:0/1703258085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1006e0a0 msgr2=0x7f7e10156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1006e0a0 0x7f7e10156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 -- 172.21.15.71:0/1703258085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e10157c80 msgr2=0x7f7e101ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e10157c80 0x7f7e101ea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 -- 172.21.15.71:0/1703258085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e100bfd40 con 0x7f7e100ac5c0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0f7fe640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e101e9bc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c7f0 tx=0x7f7df800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0cff9640 1 -- 172.21.15.71:0/1703258085 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8015070 con 0x7f7e100ac5c0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0cff9640 1 -- 172.21.15.71:0/1703258085 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df8004220 con 0x7f7e100ac5c0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e101ed130 con 0x7f7e100ac5c0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e0cff9640 1 -- 172.21.15.71:0/1703258085 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8007620 con 0x7f7e100ac5c0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.576+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e101ed6f0 con 0x7f7e100ac5c0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.578+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e10068be0 con 0x7f7e100ac5c0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.578+0000 7f7e0cff9640 1 -- 172.21.15.71:0/1703258085 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7df80079d0 con 0x7f7e100ac5c0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.578+0000 7f7e0cff9640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7df006c8f0 0x7f7df006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.578+0000 7f7e0ffff640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7df006c8f0 0x7f7df006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.578+0000 7f7e0cff9640 1 -- 172.21.15.71:0/1703258085 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7df808be00 con 0x7f7e100ac5c0 2024-04-16T15:40:34.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.579+0000 7f7e0ffff640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7df006c8f0 0x7f7df006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e00036000 tx=0x7f7e0003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.583+0000 7f7e0cff9640 1 -- 172.21.15.71:0/1703258085 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7df8059930 con 0x7f7e100ac5c0 2024-04-16T15:40:34.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.758+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e100c8c10 con 0x7f7e100ac5c0 2024-04-16T15:40:34.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.758+0000 7f7e0cff9640 1 -- 172.21.15.71:0/1703258085 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f7df805dc30 con 0x7f7e100ac5c0 2024-04-16T15:40:34.757 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:34.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7df006c8f0 msgr2=0x7f7df006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7df006c8f0 0x7f7df006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e00036000 tx=0x7f7e0003d040 comp rx=0 tx=0).stop 2024-04-16T15:40:34.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 msgr2=0x7f7e101e9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e101e9bc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c7f0 tx=0x7f7df800ccb0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 shutdown_connections 2024-04-16T15:40:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7df006c8f0 0x7f7df006edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e10157c80 0x7f7e101ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e100ac5c0 0x7f7e101e9bc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 --2- 172.21.15.71:0/1703258085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e1006e0a0 0x7f7e10156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 >> 172.21.15.71:0/1703258085 conn(0x7f7e100c8470 msgr2=0x7f7e100703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 shutdown_connections 2024-04-16T15:40:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.760+0000 7f7e166c8640 1 -- 172.21.15.71:0/1703258085 wait complete. 2024-04-16T15:40:34.770 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:40:34.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.878+0000 7fdf9a59d640 1 Processor -- start 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.878+0000 7fdf9a59d640 1 -- start start 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.878+0000 7fdf9a59d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 0x7fdf940accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.878+0000 7fdf9a59d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdf940ad210 0x7fdf940aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.878+0000 7fdf9a59d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fdf940ab950 0x7fdf94156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.878+0000 7fdf9a59d640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fdf94153ba0 con 0x7fdf940ad210 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.878+0000 7fdf9a59d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fdf94153900 con 0x7fdf940ab950 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.878+0000 7fdf9a59d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdf940ac4d0 con 0x7fdf94150f40 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf937fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdf940ad210 0x7fdf940aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf937fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdf940ad210 0x7fdf940aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49606/0 (socket says 172.21.15.71:49606) 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf937fe640 1 -- 172.21.15.71:0/2096828136 learned_addr learned my addr 172.21.15.71:0/2096828136 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf93fff640 1 --2- 172.21.15.71:0/2096828136 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 0x7fdf940accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf98b3a640 1 --2- 172.21.15.71:0/2096828136 >> v2:172.21.15.122:3300/0 conn(0x7fdf940ab950 0x7fdf94156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf937fe640 1 -- 172.21.15.71:0/2096828136 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 msgr2=0x7fdf940accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf937fe640 1 --2- 172.21.15.71:0/2096828136 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 0x7fdf940accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf937fe640 1 -- 172.21.15.71:0/2096828136 >> v2:172.21.15.122:3300/0 conn(0x7fdf940ab950 msgr2=0x7fdf94156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf937fe640 1 --2- 172.21.15.71:0/2096828136 >> v2:172.21.15.122:3300/0 conn(0x7fdf940ab950 0x7fdf94156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.879+0000 7fdf937fe640 1 -- 172.21.15.71:0/2096828136 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf940ac7b0 con 0x7fdf940ad210 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf937fe640 1 --2- 172.21.15.71:0/2096828136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf940ad210 0x7fdf940aa3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf940c2650 tx=0x7fdf8802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a075efe0bbf34d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf92ffd640 1 -- 172.21.15.71:0/2096828136 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88034070 con 0x7fdf940ad210 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf92ffd640 1 -- 172.21.15.71:0/2096828136 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf88030860 con 0x7fdf940ad210 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf92ffd640 1 -- 172.21.15.71:0/2096828136 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88030c30 con 0x7fdf940ad210 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 -- 172.21.15.71:0/2096828136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf940ad210 msgr2=0x7fdf940aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/2096828136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf940ad210 0x7fdf940aa3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf940c2650 tx=0x7fdf8802b9a0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 -- 172.21.15.71:0/2096828136 shutdown_connections 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/2096828136 >> v2:172.21.15.122:3300/0 conn(0x7fdf940ab950 0x7fdf94156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/2096828136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf940ad210 0x7fdf940aa3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/2096828136 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 0x7fdf940accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 -- 172.21.15.71:0/2096828136 >> 172.21.15.71:0/2096828136 conn(0x7fdf940c3e10 msgr2=0x7fdf940c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:34.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 -- 172.21.15.71:0/2096828136 shutdown_connections 2024-04-16T15:40:34.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.880+0000 7fdf9a59d640 1 -- 172.21.15.71:0/2096828136 wait complete. 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.881+0000 7fdf9a59d640 1 Processor -- start 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf9a59d640 1 -- start start 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf9a59d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf940ab950 0x7fdf9406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf9a59d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 0x7fdf94148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf9a59d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf941498b0 0x7fdf941491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf9a59d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9406ff50 con 0x7fdf941498b0 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf9a59d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9406fdd0 con 0x7fdf940ab950 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf9a59d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdf940c1840 con 0x7fdf94150f40 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf98b3a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf941498b0 0x7fdf941491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf98b3a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf941498b0 0x7fdf941491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49612/0 (socket says 172.21.15.71:49612) 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.882+0000 7fdf98b3a640 1 -- 172.21.15.71:0/3675238820 learned_addr learned my addr 172.21.15.71:0/3675238820 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:34.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf937fe640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 0x7fdf94148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf93fff640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf940ab950 0x7fdf9406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf98b3a640 1 -- 172.21.15.71:0/3675238820 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 msgr2=0x7fdf94148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf98b3a640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 0x7fdf94148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf98b3a640 1 -- 172.21.15.71:0/3675238820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf940ab950 msgr2=0x7fdf9406c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf98b3a640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf940ab950 0x7fdf9406c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf98b3a640 1 -- 172.21.15.71:0/3675238820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf940c58a0 con 0x7fdf941498b0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf98b3a640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf941498b0 0x7fdf941491a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c007e60 tx=0x7fdf8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf90ff9640 1 -- 172.21.15.71:0/3675238820 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c014620 con 0x7fdf941498b0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf90ff9640 1 -- 172.21.15.71:0/3675238820 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c019070 con 0x7fdf941498b0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf940c5b80 con 0x7fdf941498b0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf90ff9640 1 -- 172.21.15.71:0/3675238820 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c014a00 con 0x7fdf941498b0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.883+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf941528f0 con 0x7fdf941498b0 2024-04-16T15:40:34.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.884+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf94152af0 con 0x7fdf941498b0 2024-04-16T15:40:34.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.885+0000 7fdf90ff9640 1 -- 172.21.15.71:0/3675238820 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf8c014ba0 con 0x7fdf941498b0 2024-04-16T15:40:34.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.885+0000 7fdf90ff9640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf8006c8f0 0x7fdf8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:34.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.885+0000 7fdf90ff9640 1 -- 172.21.15.71:0/3675238820 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdf8c08c730 con 0x7fdf941498b0 2024-04-16T15:40:34.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.885+0000 7fdf93fff640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf8006c8f0 0x7fdf8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:34.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.885+0000 7fdf93fff640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf8006c8f0 0x7fdf8006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006fd0 tx=0x7fdf84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:34.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:34.889+0000 7fdf90ff9640 1 -- 172.21.15.71:0/3675238820 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf94152af0 con 0x7fdf941498b0 2024-04-16T15:40:35.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.069+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdf94068be0 con 0x7fdf941498b0 2024-04-16T15:40:35.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.069+0000 7fdf90ff9640 1 -- 172.21.15.71:0/3675238820 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fdf8c011260 con 0x7fdf941498b0 2024-04-16T15:40:35.068 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.071+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf8006c8f0 msgr2=0x7fdf8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.071+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf8006c8f0 0x7fdf8006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006fd0 tx=0x7fdf84008040 comp rx=0 tx=0).stop 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.071+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf941498b0 msgr2=0x7fdf941491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.071+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf941498b0 0x7fdf941491a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c007e60 tx=0x7fdf8c00e4b0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.072+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 shutdown_connections 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.072+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf8006c8f0 0x7fdf8006edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.072+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf941498b0 0x7fdf941491a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.072+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdf94150f40 0x7fdf94148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.072+0000 7fdf9a59d640 1 --2- 172.21.15.71:0/3675238820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf940ab950 0x7fdf9406c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.072+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 >> 172.21.15.71:0/3675238820 conn(0x7fdf940c3e10 msgr2=0x7fdf9406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.072+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 shutdown_connections 2024-04-16T15:40:35.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.072+0000 7fdf9a59d640 1 -- 172.21.15.71:0/3675238820 wait complete. 2024-04-16T15:40:35.082 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:40:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.189+0000 7fe9ecbaf640 1 Processor -- start 2024-04-16T15:40:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.189+0000 7fe9ecbaf640 1 -- start start 2024-04-16T15:40:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ecbaf640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe9e8156550 0x7fe9e8153d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ecbaf640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e806cec0 0x7fe9e8154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ecbaf640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9e806cc60 con 0x7fe9e814ffe0 2024-04-16T15:40:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ecbaf640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fe9e80c2720 con 0x7fe9e8156550 2024-04-16T15:40:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ecbaf640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e80c2fc0 con 0x7fe9e806cec0 2024-04-16T15:40:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ddd9b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe9e8156550 0x7fe9e8153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9e659c640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fe9e814ffe0 0x7fe9e8155e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56076/0 (socket says 172.21.15.71:56076) 2024-04-16T15:40:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9e659c640 1 -- 172.21.15.71:0/750606808 learned_addr learned my addr 172.21.15.71:0/750606808 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9e6d9d640 1 --2- 172.21.15.71:0/750606808 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e806cec0 0x7fe9e8154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ddd9b640 1 -- 172.21.15.71:0/750606808 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e806cec0 msgr2=0x7fe9e8154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ddd9b640 1 --2- 172.21.15.71:0/750606808 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e806cec0 0x7fe9e8154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ddd9b640 1 -- 172.21.15.71:0/750606808 >> v1:172.21.15.71:6789/0 conn(0x7fe9e814ffe0 legacy=0x7fe9e8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9ddd9b640 1 -- 172.21.15.71:0/750606808 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e81569f0 con 0x7fe9e8156550 2024-04-16T15:40:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.190+0000 7fe9e5d9b640 1 -- 172.21.15.71:0/750606808 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2676559678 0 0) 0x7fe9e806cc60 con 0x7fe9e814ffe0 2024-04-16T15:40:35.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.191+0000 7fe9ddd9b640 1 --2- 172.21.15.71:0/750606808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8153d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00097a0 tx=0x7fe9d002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0668835755a88b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.191+0000 7fe9e5d9b640 1 -- 172.21.15.71:0/750606808 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00315a0 con 0x7fe9e8156550 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.191+0000 7fe9e5d9b640 1 -- 172.21.15.71:0/750606808 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d0037070 con 0x7fe9e8156550 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.191+0000 7fe9e5d9b640 1 -- 172.21.15.71:0/750606808 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0031960 con 0x7fe9e8156550 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.191+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/750606808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 msgr2=0x7fe9e8153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.191+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/750606808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8153d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00097a0 tx=0x7fe9d002b840 comp rx=0 tx=0).stop 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.192+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/750606808 shutdown_connections 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.192+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/750606808 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e806cec0 0x7fe9e8154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.192+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/750606808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8153d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.192+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/750606808 >> 172.21.15.71:0/750606808 conn(0x7fe9e80c7940 msgr2=0x7fe9e80c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.192+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/750606808 shutdown_connections 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.192+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/750606808 wait complete. 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.193+0000 7fe9ecbaf640 1 Processor -- start 2024-04-16T15:40:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.193+0000 7fe9ecbaf640 1 -- start start 2024-04-16T15:40:35.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ecbaf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9e806cec0 0x7fe9e806bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ecbaf640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e814ffe0 0x7fe9e8141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ecbaf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ecbaf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e80c4740 con 0x7fe9e8156550 2024-04-16T15:40:35.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ecbaf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e80c2c50 con 0x7fe9e806cec0 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ecbaf640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e80c3450 con 0x7fe9e814ffe0 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8141a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49630/0 (socket says 172.21.15.71:49630) 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 -- 172.21.15.71:0/381728657 learned_addr learned my addr 172.21.15.71:0/381728657 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ddd9b640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e814ffe0 0x7fe9e8141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e659c640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9e806cec0 0x7fe9e806bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 -- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e814ffe0 msgr2=0x7fe9e8141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e814ffe0 0x7fe9e8141510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 -- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9e806cec0 msgr2=0x7fe9e806bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9e806cec0 0x7fe9e806bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 -- 172.21.15.71:0/381728657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e8150bf0 con 0x7fe9e8156550 2024-04-16T15:40:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ddd9b640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e814ffe0 0x7fe9e8141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:35.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e659c640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9e806cec0 0x7fe9e806bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:35.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9e6d9d640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8141a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0007de0 tx=0x7fe9e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9df7fe640 1 -- 172.21.15.71:0/381728657 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0014620 con 0x7fe9e8156550 2024-04-16T15:40:35.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9df7fe640 1 -- 172.21.15.71:0/381728657 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e0019070 con 0x7fe9e8156550 2024-04-16T15:40:35.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e80c6ae0 con 0x7fe9e8156550 2024-04-16T15:40:35.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9df7fe640 1 -- 172.21.15.71:0/381728657 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0014a20 con 0x7fe9e8156550 2024-04-16T15:40:35.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.194+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e806c0e0 con 0x7fe9e8156550 2024-04-16T15:40:35.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.196+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e80c6cb0 con 0x7fe9e8156550 2024-04-16T15:40:35.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.196+0000 7fe9df7fe640 1 -- 172.21.15.71:0/381728657 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9e0014bc0 con 0x7fe9e8156550 2024-04-16T15:40:35.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.196+0000 7fe9df7fe640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe9c806c8f0 0x7fe9c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.196+0000 7fe9df7fe640 1 -- 172.21.15.71:0/381728657 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe9e008c7e0 con 0x7fe9e8156550 2024-04-16T15:40:35.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.196+0000 7fe9e659c640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe9c806c8f0 0x7fe9c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.197+0000 7fe9e659c640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe9c806c8f0 0x7fe9c806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8000f60 tx=0x7fe9d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.200+0000 7fe9df7fe640 1 -- 172.21.15.71:0/381728657 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9e8150bf0 con 0x7fe9e8156550 2024-04-16T15:40:35.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.376+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe9e8068be0 con 0x7fe9e8156550 2024-04-16T15:40:35.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.376+0000 7fe9df7fe640 1 -- 172.21.15.71:0/381728657 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fe9e005a8d0 con 0x7fe9e8156550 2024-04-16T15:40:35.375 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:40:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe9c806c8f0 msgr2=0x7fe9c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe9c806c8f0 0x7fe9c806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8000f60 tx=0x7fe9d8009040 comp rx=0 tx=0).stop 2024-04-16T15:40:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 msgr2=0x7fe9e8141a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8141a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0007de0 tx=0x7fe9e000e4b0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 shutdown_connections 2024-04-16T15:40:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe9c806c8f0 0x7fe9c806edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8156550 0x7fe9e8141a50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe9e814ffe0 0x7fe9e8141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 --2- 172.21.15.71:0/381728657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9e806cec0 0x7fe9e806bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 >> 172.21.15.71:0/381728657 conn(0x7fe9e80c7940 msgr2=0x7fe9e814add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 shutdown_connections 2024-04-16T15:40:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.378+0000 7fe9ecbaf640 1 -- 172.21.15.71:0/381728657 wait complete. 2024-04-16T15:40:35.388 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:40:35.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c39835640 1 Processor -- start 2024-04-16T15:40:35.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c39835640 1 -- start start 2024-04-16T15:40:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c39835640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340c6680 0x7f0c34153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c39835640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0c340be520 0x7f0c340be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c39835640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0c340bee80 0x7f0c340c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c39835640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f0c340c0760 con 0x7f0c340be520 2024-04-16T15:40:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c39835640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f0c340c80b0 con 0x7f0c340bee80 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c39835640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0c340c74a0 con 0x7f0c340c6680 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.496+0000 7f0c327fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0c340be520 0x7f0c340be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c327fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0c340be520 0x7f0c340be940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49636/0 (socket says 172.21.15.71:49636) 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c327fc640 1 -- 172.21.15.71:0/3168560381 learned_addr learned my addr 172.21.15.71:0/3168560381 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c32ffd640 1 --2- 172.21.15.71:0/3168560381 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340c6680 0x7f0c34153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c337fe640 1 --2- 172.21.15.71:0/3168560381 >> v2:172.21.15.122:3300/0 conn(0x7f0c340bee80 0x7f0c340c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c327fc640 1 -- 172.21.15.71:0/3168560381 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340c6680 msgr2=0x7f0c34153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c327fc640 1 --2- 172.21.15.71:0/3168560381 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340c6680 0x7f0c34153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c327fc640 1 -- 172.21.15.71:0/3168560381 >> v2:172.21.15.122:3300/0 conn(0x7f0c340bee80 msgr2=0x7f0c340c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c327fc640 1 --2- 172.21.15.71:0/3168560381 >> v2:172.21.15.122:3300/0 conn(0x7f0c340bee80 0x7f0c340c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c327fc640 1 -- 172.21.15.71:0/3168560381 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c340c7780 con 0x7f0c340be520 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.497+0000 7f0c337fe640 1 --2- 172.21.15.71:0/3168560381 >> v2:172.21.15.122:3300/0 conn(0x7f0c340bee80 0x7f0c340c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c327fc640 1 --2- 172.21.15.71:0/3168560381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340be520 0x7f0c340be940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25d536f10af535b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c31ffb640 1 -- 172.21.15.71:0/3168560381 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0317c0 con 0x7f0c340be520 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c31ffb640 1 -- 172.21.15.71:0/3168560381 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c037070 con 0x7f0c340be520 2024-04-16T15:40:35.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c31ffb640 1 -- 172.21.15.71:0/3168560381 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c031b80 con 0x7f0c340be520 2024-04-16T15:40:35.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 -- 172.21.15.71:0/3168560381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340be520 msgr2=0x7f0c340be940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 --2- 172.21.15.71:0/3168560381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340be520 0x7f0c340be940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c02b860 comp rx=0 tx=0).stop 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 -- 172.21.15.71:0/3168560381 shutdown_connections 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 --2- 172.21.15.71:0/3168560381 >> v2:172.21.15.122:3300/0 conn(0x7f0c340bee80 0x7f0c340c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 --2- 172.21.15.71:0/3168560381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340be520 0x7f0c340be940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 --2- 172.21.15.71:0/3168560381 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340c6680 0x7f0c34153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 -- 172.21.15.71:0/3168560381 >> 172.21.15.71:0/3168560381 conn(0x7f0c340c96e0 msgr2=0x7f0c340c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 -- 172.21.15.71:0/3168560381 shutdown_connections 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.498+0000 7f0c39835640 1 -- 172.21.15.71:0/3168560381 wait complete. 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.499+0000 7f0c39835640 1 Processor -- start 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c39835640 1 -- start start 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c39835640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340be520 0x7f0c34147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c39835640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340bee80 0x7f0c34149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c39835640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c340c6680 0x7f0c3414fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c39835640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c340c4120 con 0x7f0c340bee80 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c39835640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c340c7d40 con 0x7f0c340c6680 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c39835640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0c340c5240 con 0x7f0c340be520 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c327fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340bee80 0x7f0c34149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c327fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340bee80 0x7f0c34149590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49646/0 (socket says 172.21.15.71:49646) 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.500+0000 7f0c327fc640 1 -- 172.21.15.71:0/3200259777 learned_addr learned my addr 172.21.15.71:0/3200259777 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c337fe640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c340c6680 0x7f0c3414fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c32ffd640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340be520 0x7f0c34147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c327fc640 1 -- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340be520 msgr2=0x7f0c34147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c327fc640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340be520 0x7f0c34147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c327fc640 1 -- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c340c6680 msgr2=0x7f0c3414fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c327fc640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c340c6680 0x7f0c3414fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c327fc640 1 -- 172.21.15.71:0/3200259777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c341e10a0 con 0x7f0c340bee80 2024-04-16T15:40:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c32ffd640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340be520 0x7f0c34147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c327fc640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340bee80 0x7f0c34149590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c000c00 tx=0x7f0c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c2bfff640 1 -- 172.21.15.71:0/3200259777 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c032650 con 0x7f0c340bee80 2024-04-16T15:40:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c2bfff640 1 -- 172.21.15.71:0/3200259777 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0327f0 con 0x7f0c340bee80 2024-04-16T15:40:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c2bfff640 1 -- 172.21.15.71:0/3200259777 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c032be0 con 0x7f0c340bee80 2024-04-16T15:40:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c341e1380 con 0x7f0c340bee80 2024-04-16T15:40:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.501+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c34149ad0 con 0x7f0c340bee80 2024-04-16T15:40:35.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.502+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c34068be0 con 0x7f0c340bee80 2024-04-16T15:40:35.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.503+0000 7f0c2bfff640 1 -- 172.21.15.71:0/3200259777 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c1c04c060 con 0x7f0c340bee80 2024-04-16T15:40:35.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.503+0000 7f0c2bfff640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0c1406c8f0 0x7f0c1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.503+0000 7f0c32ffd640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0c1406c8f0 0x7f0c1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.503+0000 7f0c2bfff640 1 -- 172.21.15.71:0/3200259777 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0c1c0ad3f0 con 0x7f0c340bee80 2024-04-16T15:40:35.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.504+0000 7f0c32ffd640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0c1406c8f0 0x7f0c1406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0c341df640 tx=0x7f0c2400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.508+0000 7f0c2bfff640 1 -- 172.21.15.71:0/3200259777 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c1c040070 con 0x7f0c340bee80 2024-04-16T15:40:35.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.681+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0c340bf4a0 con 0x7f0c340bee80 2024-04-16T15:40:35.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.681+0000 7f0c2bfff640 1 -- 172.21.15.71:0/3200259777 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f0c1c07b570 con 0x7f0c340bee80 2024-04-16T15:40:35.680 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0c1406c8f0 msgr2=0x7f0c1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0c1406c8f0 0x7f0c1406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0c341df640 tx=0x7f0c2400b040 comp rx=0 tx=0).stop 2024-04-16T15:40:35.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340bee80 msgr2=0x7f0c34149590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340bee80 0x7f0c34149590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c000c00 tx=0x7f0c1c0046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 shutdown_connections 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0c1406c8f0 0x7f0c1406edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c340c6680 0x7f0c3414fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c340bee80 0x7f0c34149590 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 --2- 172.21.15.71:0/3200259777 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0c340be520 0x7f0c34147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.683+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 >> 172.21.15.71:0/3200259777 conn(0x7f0c340c96e0 msgr2=0x7f0c340c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.684+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 shutdown_connections 2024-04-16T15:40:35.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.684+0000 7f0c39835640 1 -- 172.21.15.71:0/3200259777 wait complete. 2024-04-16T15:40:35.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T15:40:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.799+0000 7fbb8db50640 1 Processor -- start 2024-04-16T15:40:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.799+0000 7fbb8db50640 1 -- start start 2024-04-16T15:40:35.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.800+0000 7fbb8db50640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fbb8806d920 0x7fbb88150d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.800+0000 7fbb8db50640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb880a9620 con 0x7fbb88151270 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.800+0000 7fbb8db50640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb880a98e0 con 0x7fbb8806d110 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.800+0000 7fbb8db50640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fbb8806c570 con 0x7fbb8806d920 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.800+0000 7fbb87fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fbb88151270 0x7fbb880729f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56086/0 (socket says 172.21.15.71:56086) 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.800+0000 7fbb87fff640 1 -- 172.21.15.71:0/600217469 learned_addr learned my addr 172.21.15.71:0/600217469 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.800+0000 7fbb7effd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fbb8806d920 0x7fbb88150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.801+0000 7fbb7effd640 1 -- 172.21.15.71:0/600217469 >> v1:172.21.15.122:6789/0 conn(0x7fbb8806d110 legacy=0x7fbb88154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.801+0000 7fbb7effd640 1 -- 172.21.15.71:0/600217469 >> v1:172.21.15.71:6789/0 conn(0x7fbb88151270 legacy=0x7fbb880729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.801+0000 7fbb7effd640 1 -- 172.21.15.71:0/600217469 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb880a9bc0 con 0x7fbb8806d920 2024-04-16T15:40:35.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.801+0000 7fbb86ffd640 1 -- 172.21.15.71:0/600217469 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509818567 0 0) 0x7fbb880a9620 con 0x7fbb88151270 2024-04-16T15:40:35.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.801+0000 7fbb7effd640 1 --2- 172.21.15.71:0/600217469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb8806d920 0x7fbb88150d30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097a0 tx=0x7fbb7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e051fb54ce7ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb86ffd640 1 -- 172.21.15.71:0/600217469 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70035020 con 0x7fbb8806d920 2024-04-16T15:40:35.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb86ffd640 1 -- 172.21.15.71:0/600217469 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb70031830 con 0x7fbb8806d920 2024-04-16T15:40:35.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb86ffd640 1 -- 172.21.15.71:0/600217469 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70031c00 con 0x7fbb8806d920 2024-04-16T15:40:35.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb8db50640 1 -- 172.21.15.71:0/600217469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb8806d920 msgr2=0x7fbb88150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb8db50640 1 --2- 172.21.15.71:0/600217469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb8806d920 0x7fbb88150d30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097a0 tx=0x7fbb7002b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb8db50640 1 -- 172.21.15.71:0/600217469 shutdown_connections 2024-04-16T15:40:35.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb8db50640 1 --2- 172.21.15.71:0/600217469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb8806d920 0x7fbb88150d30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb8db50640 1 -- 172.21.15.71:0/600217469 >> 172.21.15.71:0/600217469 conn(0x7fbb8806e6e0 msgr2=0x7fbb8806eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:35.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb8db50640 1 -- 172.21.15.71:0/600217469 shutdown_connections 2024-04-16T15:40:35.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.802+0000 7fbb8db50640 1 -- 172.21.15.71:0/600217469 wait complete. 2024-04-16T15:40:35.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.804+0000 7fbb8db50640 1 Processor -- start 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.804+0000 7fbb8db50640 1 -- start start 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.804+0000 7fbb8db50640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb8806d110 0x7fbb880c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.804+0000 7fbb8db50640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb8806d920 0x7fbb88141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.804+0000 7fbb8db50640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb88151270 0x7fbb8814a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.804+0000 7fbb8db50640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8806d4f0 con 0x7fbb88151270 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.804+0000 7fbb8db50640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb880aa210 con 0x7fbb8806d110 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.804+0000 7fbb8db50640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8806d670 con 0x7fbb8806d920 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb88151270 0x7fbb8814a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb88151270 0x7fbb8814a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49660/0 (socket says 172.21.15.71:49660) 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 -- 172.21.15.71:0/2964055706 learned_addr learned my addr 172.21.15.71:0/2964055706 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb7effd640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb8806d920 0x7fbb88141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb877fe640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb8806d110 0x7fbb880c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 -- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb8806d920 msgr2=0x7fbb88141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb8806d920 0x7fbb88141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 -- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb8806d110 msgr2=0x7fbb880c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb8806d110 0x7fbb880c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 -- 172.21.15.71:0/2964055706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb881415a0 con 0x7fbb88151270 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb7effd640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb8806d920 0x7fbb88141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb877fe640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb8806d110 0x7fbb880c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb87fff640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb88151270 0x7fbb8814a1f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c730 tx=0x7fbb8000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb84ff9640 1 -- 172.21.15.71:0/2964055706 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80018070 con 0x7fbb88151270 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb84ff9640 1 -- 172.21.15.71:0/2964055706 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8000ce40 con 0x7fbb88151270 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb880c65e0 con 0x7fbb88151270 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb84ff9640 1 -- 172.21.15.71:0/2964055706 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000a860 con 0x7fbb88151270 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.805+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb881ecf50 con 0x7fbb88151270 2024-04-16T15:40:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.806+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8806ef30 con 0x7fbb88151270 2024-04-16T15:40:35.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.807+0000 7fbb84ff9640 1 -- 172.21.15.71:0/2964055706 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb8000aa00 con 0x7fbb88151270 2024-04-16T15:40:35.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.807+0000 7fbb84ff9640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbb6806c8f0 0x7fbb6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:35.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.807+0000 7fbb84ff9640 1 -- 172.21.15.71:0/2964055706 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbb8008b480 con 0x7fbb88151270 2024-04-16T15:40:35.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.807+0000 7fbb877fe640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbb6806c8f0 0x7fbb6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:35.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.808+0000 7fbb877fe640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbb6806c8f0 0x7fbb6806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb8814b0c0 tx=0x7fbb78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:35.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.811+0000 7fbb84ff9640 1 -- 172.21.15.71:0/2964055706 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb8806ef30 con 0x7fbb88151270 2024-04-16T15:40:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:35.999+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb881de670 con 0x7fbb88151270 2024-04-16T15:40:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.000+0000 7fbb84ff9640 1 -- 172.21.15.71:0/2964055706 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fbb881415a0 con 0x7fbb88151270 2024-04-16T15:40:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-04-16T15:40:36.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.001+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbb6806c8f0 msgr2=0x7fbb6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.001+0000 7fbb8db50640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbb6806c8f0 0x7fbb6806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb8814b0c0 tx=0x7fbb78009040 comp rx=0 tx=0).stop 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.001+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb88151270 msgr2=0x7fbb8814a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.001+0000 7fbb8db50640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb88151270 0x7fbb8814a1f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c730 tx=0x7fbb8000cbf0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.002+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 shutdown_connections 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.002+0000 7fbb8db50640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbb6806c8f0 0x7fbb6806edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.002+0000 7fbb8db50640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb88151270 0x7fbb8814a1f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.002+0000 7fbb8db50640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbb8806d920 0x7fbb88141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.002+0000 7fbb8db50640 1 --2- 172.21.15.71:0/2964055706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb8806d110 0x7fbb880c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.002+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 >> 172.21.15.71:0/2964055706 conn(0x7fbb8806e6e0 msgr2=0x7fbb88142c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.002+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 shutdown_connections 2024-04-16T15:40:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.002+0000 7fbb8db50640 1 -- 172.21.15.71:0/2964055706 wait complete. 2024-04-16T15:40:36.011 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:40:36.011 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:40:36.028 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.171+0000 7facd242a640 1 Processor -- start 2024-04-16T15:40:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.171+0000 7facd242a640 1 -- start start 2024-04-16T15:40:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7facd242a640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7faccc155dc0 0x7faccc0c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7facd242a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7faccc06fac0 0x7faccc1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7facd242a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc0ab290 0x7faccc1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7facd242a640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7faccc073f20 con 0x7faccc155dc0 2024-04-16T15:40:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7facd242a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7faccc156ab0 con 0x7faccc06fac0 2024-04-16T15:40:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7facd242a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7faccc0c27c0 con 0x7faccc0ab290 2024-04-16T15:40:36.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7faccbfff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7faccc155dc0 0x7faccc0c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7faccbfff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7faccc155dc0 0x7faccc0c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49664/0 (socket says 172.21.15.71:49664) 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7faccbfff640 1 -- 172.21.15.71:0/3345789486 learned_addr learned my addr 172.21.15.71:0/3345789486 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7facd09c7640 1 --2- 172.21.15.71:0/3345789486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc0ab290 0x7faccc1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.172+0000 7facc3fff640 1 --2- 172.21.15.71:0/3345789486 >> v2:172.21.15.122:3300/0 conn(0x7faccc06fac0 0x7faccc1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccbfff640 1 -- 172.21.15.71:0/3345789486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc0ab290 msgr2=0x7faccc1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccbfff640 1 --2- 172.21.15.71:0/3345789486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc0ab290 0x7faccc1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccbfff640 1 -- 172.21.15.71:0/3345789486 >> v2:172.21.15.122:3300/0 conn(0x7faccc06fac0 msgr2=0x7faccc1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccbfff640 1 --2- 172.21.15.71:0/3345789486 >> v2:172.21.15.122:3300/0 conn(0x7faccc06fac0 0x7faccc1561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccbfff640 1 -- 172.21.15.71:0/3345789486 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc0c2aa0 con 0x7faccc155dc0 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7facd09c7640 1 --2- 172.21.15.71:0/3345789486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc0ab290 0x7faccc1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7facc3fff640 1 --2- 172.21.15.71:0/3345789486 >> v2:172.21.15.122:3300/0 conn(0x7faccc06fac0 0x7faccc1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccbfff640 1 --2- 172.21.15.71:0/3345789486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc155dc0 0x7faccc0c32a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faccc06d2d0 tx=0x7facbc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01c6fae556103b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccb7fe640 1 -- 172.21.15.71:0/3345789486 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc035020 con 0x7faccc155dc0 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccb7fe640 1 -- 172.21.15.71:0/3345789486 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc031870 con 0x7faccc155dc0 2024-04-16T15:40:36.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.173+0000 7faccb7fe640 1 -- 172.21.15.71:0/3345789486 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc031c40 con 0x7faccc155dc0 2024-04-16T15:40:36.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 -- 172.21.15.71:0/3345789486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc155dc0 msgr2=0x7faccc0c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 --2- 172.21.15.71:0/3345789486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc155dc0 0x7faccc0c32a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faccc06d2d0 tx=0x7facbc02b910 comp rx=0 tx=0).stop 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 -- 172.21.15.71:0/3345789486 shutdown_connections 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 --2- 172.21.15.71:0/3345789486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc0ab290 0x7faccc1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 --2- 172.21.15.71:0/3345789486 >> v2:172.21.15.122:3300/0 conn(0x7faccc06fac0 0x7faccc1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 --2- 172.21.15.71:0/3345789486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc155dc0 0x7faccc0c32a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 -- 172.21.15.71:0/3345789486 >> 172.21.15.71:0/3345789486 conn(0x7faccc0c3e10 msgr2=0x7faccc0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 -- 172.21.15.71:0/3345789486 shutdown_connections 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.174+0000 7facd242a640 1 -- 172.21.15.71:0/3345789486 wait complete. 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.175+0000 7facd242a640 1 Processor -- start 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.175+0000 7facd242a640 1 -- start start 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7facd242a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc0ab290 0x7faccc153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7facd242a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faccc155dc0 0x7faccc154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7facd242a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc13cba0 0x7faccc13d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7facd242a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1566e0 con 0x7faccc0ab290 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7facd242a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faccc0c18b0 con 0x7faccc155dc0 2024-04-16T15:40:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7facd242a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7faccc156860 con 0x7faccc13cba0 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc0ab290 0x7faccc153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc0ab290 0x7faccc153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49668/0 (socket says 172.21.15.71:49668) 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 -- 172.21.15.71:0/642030293 learned_addr learned my addr 172.21.15.71:0/642030293 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7facd09c7640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc13cba0 0x7faccc13d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7facc3fff640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faccc155dc0 0x7faccc154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 -- 172.21.15.71:0/642030293 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc13cba0 msgr2=0x7faccc13d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc13cba0 0x7faccc13d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 -- 172.21.15.71:0/642030293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faccc155dc0 msgr2=0x7faccc154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faccc155dc0 0x7faccc154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 -- 172.21.15.71:0/642030293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc1541a0 con 0x7faccc0ab290 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.176+0000 7faccbfff640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc0ab290 0x7faccc153bc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7facbc02b950 tx=0x7facbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.177+0000 7facc97fa640 1 -- 172.21.15.71:0/642030293 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc040070 con 0x7faccc0ab290 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.177+0000 7facc97fa640 1 -- 172.21.15.71:0/642030293 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facbc030d00 con 0x7faccc0ab290 2024-04-16T15:40:36.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.177+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc1dcfe0 con 0x7faccc0ab290 2024-04-16T15:40:36.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.177+0000 7facc97fa640 1 -- 172.21.15.71:0/642030293 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc035040 con 0x7faccc0ab290 2024-04-16T15:40:36.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.177+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc1551f0 con 0x7faccc0ab290 2024-04-16T15:40:36.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.178+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faccc1dd6a0 con 0x7faccc0ab290 2024-04-16T15:40:36.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.178+0000 7facc97fa640 1 -- 172.21.15.71:0/642030293 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facbc038020 con 0x7faccc0ab290 2024-04-16T15:40:36.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.179+0000 7facc97fa640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facac06c8f0 0x7facac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.179+0000 7facc3fff640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facac06c8f0 0x7facac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.179+0000 7facc97fa640 1 -- 172.21.15.71:0/642030293 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7facbc0adb50 con 0x7faccc0ab290 2024-04-16T15:40:36.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.179+0000 7facc3fff640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facac06c8f0 0x7facac06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7facb4006fd0 tx=0x7facb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.183+0000 7facc97fa640 1 -- 172.21.15.71:0/642030293 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faccc1dd6a0 con 0x7faccc0ab290 2024-04-16T15:40:36.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.359+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faccc0c35b0 con 0x7faccc0ab290 2024-04-16T15:40:36.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.360+0000 7facc97fa640 1 -- 172.21.15.71:0/642030293 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7faccc0c35b0 con 0x7faccc0ab290 2024-04-16T15:40:36.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:36.359 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:36.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facac06c8f0 msgr2=0x7facac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facac06c8f0 0x7facac06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7facb4006fd0 tx=0x7facb4008040 comp rx=0 tx=0).stop 2024-04-16T15:40:36.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc0ab290 msgr2=0x7faccc153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc0ab290 0x7faccc153bc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7facbc02b950 tx=0x7facbc0046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 shutdown_connections 2024-04-16T15:40:36.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facac06c8f0 0x7facac06edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faccc13cba0 0x7faccc13d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faccc155dc0 0x7faccc154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 --2- 172.21.15.71:0/642030293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faccc0ab290 0x7faccc153bc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 >> 172.21.15.71:0/642030293 conn(0x7faccc0c3e10 msgr2=0x7faccc1de5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:36.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 shutdown_connections 2024-04-16T15:40:36.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.362+0000 7facd242a640 1 -- 172.21.15.71:0/642030293 wait complete. 2024-04-16T15:40:36.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:40:36.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:36.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.479+0000 7f69c482a640 1 Processor -- start 2024-04-16T15:40:36.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.479+0000 7f69c482a640 1 -- start start 2024-04-16T15:40:36.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69c482a640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f69c01514b0 0x7f69c00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69c482a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c00c8ee0 0x7f69c0157300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69c482a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f69c00c4b10 0x7f69c0157840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69c482a640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f69c00be6e0 con 0x7f69c01514b0 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69c482a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f69c0157d80 con 0x7f69c00c4b10 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69c482a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f69c00c55b0 con 0x7f69c00c8ee0 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69be59c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f69c01514b0 0x7f69c00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69be59c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f69c01514b0 0x7f69c00c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49676/0 (socket says 172.21.15.71:49676) 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.480+0000 7f69be59c640 1 -- 172.21.15.71:0/2929678089 learned_addr learned my addr 172.21.15.71:0/2929678089 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69bed9d640 1 --2- 172.21.15.71:0/2929678089 >> v2:172.21.15.122:3300/0 conn(0x7f69c00c4b10 0x7f69c0157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69bdd9b640 1 --2- 172.21.15.71:0/2929678089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c00c8ee0 0x7f69c0157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69be59c640 1 -- 172.21.15.71:0/2929678089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c00c8ee0 msgr2=0x7f69c0157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69be59c640 1 --2- 172.21.15.71:0/2929678089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c00c8ee0 0x7f69c0157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69be59c640 1 -- 172.21.15.71:0/2929678089 >> v2:172.21.15.122:3300/0 conn(0x7f69c00c4b10 msgr2=0x7f69c0157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69be59c640 1 --2- 172.21.15.71:0/2929678089 >> v2:172.21.15.122:3300/0 conn(0x7f69c00c4b10 0x7f69c0157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69be59c640 1 -- 172.21.15.71:0/2929678089 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c00c5890 con 0x7f69c01514b0 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69bed9d640 1 --2- 172.21.15.71:0/2929678089 >> v2:172.21.15.122:3300/0 conn(0x7f69c00c4b10 0x7f69c0157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69bdd9b640 1 --2- 172.21.15.71:0/2929678089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c00c8ee0 0x7f69c0157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.481+0000 7f69be59c640 1 --2- 172.21.15.71:0/2929678089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c01514b0 0x7f69c00c94a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69c00ac290 tx=0x7f69b002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dd065fd429c397 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69bd59a640 1 -- 172.21.15.71:0/2929678089 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b002c9e0 con 0x7f69c01514b0 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69bd59a640 1 -- 172.21.15.71:0/2929678089 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b002cb80 con 0x7f69c01514b0 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69bd59a640 1 -- 172.21.15.71:0/2929678089 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b00343f0 con 0x7f69c01514b0 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 -- 172.21.15.71:0/2929678089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c01514b0 msgr2=0x7f69c00c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 --2- 172.21.15.71:0/2929678089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c01514b0 0x7f69c00c94a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69c00ac290 tx=0x7f69b002be40 comp rx=0 tx=0).stop 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 -- 172.21.15.71:0/2929678089 shutdown_connections 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 --2- 172.21.15.71:0/2929678089 >> v2:172.21.15.122:3300/0 conn(0x7f69c00c4b10 0x7f69c0157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 --2- 172.21.15.71:0/2929678089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c00c8ee0 0x7f69c0157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 --2- 172.21.15.71:0/2929678089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c01514b0 0x7f69c00c94a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 -- 172.21.15.71:0/2929678089 >> 172.21.15.71:0/2929678089 conn(0x7f69c00ab3b0 msgr2=0x7f69c00ab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 -- 172.21.15.71:0/2929678089 shutdown_connections 2024-04-16T15:40:36.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.482+0000 7f69c482a640 1 -- 172.21.15.71:0/2929678089 wait complete. 2024-04-16T15:40:36.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.483+0000 7f69c482a640 1 Processor -- start 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69c482a640 1 -- start start 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69c482a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c00c4b10 0x7f69c01569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69c482a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c00c8ee0 0x7f69c014c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69c482a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c01514b0 0x7f69c014c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69c482a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f69c00ae690 con 0x7f69c00c4b10 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69c482a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69c00beb70 con 0x7f69c00c8ee0 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69c482a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f69c00ae810 con 0x7f69c01514b0 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69be59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c00c4b10 0x7f69c01569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69be59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c00c4b10 0x7f69c01569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49684/0 (socket says 172.21.15.71:49684) 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69be59c640 1 -- 172.21.15.71:0/4277754506 learned_addr learned my addr 172.21.15.71:0/4277754506 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69bed9d640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c01514b0 0x7f69c014c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.484+0000 7f69bdd9b640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c00c8ee0 0x7f69c014c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69be59c640 1 -- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c01514b0 msgr2=0x7f69c014c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69be59c640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c01514b0 0x7f69c014c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69be59c640 1 -- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c00c8ee0 msgr2=0x7f69c014c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69be59c640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c00c8ee0 0x7f69c014c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69be59c640 1 -- 172.21.15.71:0/4277754506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c01ef920 con 0x7f69c00c4b10 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69bdd9b640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c00c8ee0 0x7f69c014c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69be59c640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c00c4b10 0x7f69c01569d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f69b0036040 tx=0x7f69b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69aeffd640 1 -- 172.21.15.71:0/4277754506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0043020 con 0x7f69c00c4b10 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69aeffd640 1 -- 172.21.15.71:0/4277754506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b003e450 con 0x7f69c00c4b10 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c01efc00 con 0x7f69c00c4b10 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69aeffd640 1 -- 172.21.15.71:0/4277754506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b003e820 con 0x7f69c00c4b10 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.485+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c00c2b80 con 0x7f69c00c4b10 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.486+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c0068be0 con 0x7f69c00c4b10 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.486+0000 7f69aeffd640 1 -- 172.21.15.71:0/4277754506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69b0046020 con 0x7f69c00c4b10 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.487+0000 7f69aeffd640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f69a006c8f0 0x7f69a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.487+0000 7f69aeffd640 1 -- 172.21.15.71:0/4277754506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f69b00ae510 con 0x7f69c00c4b10 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.487+0000 7f69bdd9b640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f69a006c8f0 0x7f69a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.487+0000 7f69bdd9b640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f69a006c8f0 0x7f69a006edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006fd0 tx=0x7f69b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.491+0000 7f69aeffd640 1 -- 172.21.15.71:0/4277754506 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69b004d070 con 0x7f69c00c4b10 2024-04-16T15:40:36.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.670+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69c00c3e20 con 0x7f69c00c4b10 2024-04-16T15:40:36.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.671+0000 7f69aeffd640 1 -- 172.21.15.71:0/4277754506 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f69b007c690 con 0x7f69c00c4b10 2024-04-16T15:40:36.670 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f69a006c8f0 msgr2=0x7f69a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f69a006c8f0 0x7f69a006edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006fd0 tx=0x7f69b4008040 comp rx=0 tx=0).stop 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c00c4b10 msgr2=0x7f69c01569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c00c4b10 0x7f69c01569d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f69b0036040 tx=0x7f69b0004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 shutdown_connections 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f69a006c8f0 0x7f69a006edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f69c01514b0 0x7f69c014c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69c00c8ee0 0x7f69c014c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 --2- 172.21.15.71:0/4277754506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69c00c4b10 0x7f69c01569d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 >> 172.21.15.71:0/4277754506 conn(0x7f69c00ab3b0 msgr2=0x7f69c00aa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:36.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 shutdown_connections 2024-04-16T15:40:36.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.673+0000 7f69c482a640 1 -- 172.21.15.71:0/4277754506 wait complete. 2024-04-16T15:40:36.683 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:40:36.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.791+0000 7fb57fd82640 1 Processor -- start 2024-04-16T15:40:36.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.791+0000 7fb57fd82640 1 -- start start 2024-04-16T15:40:36.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57fd82640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb578153210 0x7fb5780aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57fd82640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb57806fb20 0x7fb57806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57fd82640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5780ab030 con 0x7fb578156620 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57fd82640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fb578070490 con 0x7fb578153210 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57fd82640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb5781535f0 con 0x7fb57806fb20 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57db1e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb578153210 0x7fb5780aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57db1e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb578153210 0x7fb5780aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49686/0 (socket says 172.21.15.71:49686) 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57db1e640 1 -- 172.21.15.71:0/3778829291 learned_addr learned my addr 172.21.15.71:0/3778829291 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57e31f640 1 --2- 172.21.15.71:0/3778829291 >> v2:172.21.15.122:3300/0 conn(0x7fb57806fb20 0x7fb57806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57db1e640 1 -- 172.21.15.71:0/3778829291 >> v2:172.21.15.122:3300/0 conn(0x7fb57806fb20 msgr2=0x7fb57806ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57db1e640 1 --2- 172.21.15.71:0/3778829291 >> v2:172.21.15.122:3300/0 conn(0x7fb57806fb20 0x7fb57806ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57db1e640 1 -- 172.21.15.71:0/3778829291 >> v1:172.21.15.122:6789/0 conn(0x7fb578156620 legacy=0x7fb57806da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.792+0000 7fb57db1e640 1 -- 172.21.15.71:0/3778829291 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5780c1c10 con 0x7fb578153210 2024-04-16T15:40:36.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.793+0000 7fb57e31f640 1 --2- 172.21.15.71:0/3778829291 >> v2:172.21.15.122:3300/0 conn(0x7fb57806fb20 0x7fb57806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.793+0000 7fb57db1e640 1 --2- 172.21.15.71:0/3778829291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb578153210 0x7fb5780aa9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5780aaf60 tx=0x7fb56802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b7ae064728b7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.793+0000 7fb57cb1c640 1 -- 172.21.15.71:0/3778829291 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56802c9f0 con 0x7fb578153210 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.793+0000 7fb57cb1c640 1 -- 172.21.15.71:0/3778829291 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56802cb90 con 0x7fb578153210 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.793+0000 7fb57cb1c640 1 -- 172.21.15.71:0/3778829291 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5680343f0 con 0x7fb578153210 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.794+0000 7fb57fd82640 1 -- 172.21.15.71:0/3778829291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb578153210 msgr2=0x7fb5780aa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.794+0000 7fb57fd82640 1 --2- 172.21.15.71:0/3778829291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb578153210 0x7fb5780aa9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5780aaf60 tx=0x7fb56802be50 comp rx=0 tx=0).stop 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.794+0000 7fb57fd82640 1 -- 172.21.15.71:0/3778829291 shutdown_connections 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.794+0000 7fb57fd82640 1 --2- 172.21.15.71:0/3778829291 >> v2:172.21.15.122:3300/0 conn(0x7fb57806fb20 0x7fb57806ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.794+0000 7fb57fd82640 1 --2- 172.21.15.71:0/3778829291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb578153210 0x7fb5780aa9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.794+0000 7fb57fd82640 1 -- 172.21.15.71:0/3778829291 >> 172.21.15.71:0/3778829291 conn(0x7fb578072aa0 msgr2=0x7fb578072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.794+0000 7fb57fd82640 1 -- 172.21.15.71:0/3778829291 shutdown_connections 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.794+0000 7fb57fd82640 1 -- 172.21.15.71:0/3778829291 wait complete. 2024-04-16T15:40:36.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.795+0000 7fb57fd82640 1 Processor -- start 2024-04-16T15:40:36.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.795+0000 7fb57fd82640 1 -- start start 2024-04-16T15:40:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57fd82640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb57806fb20 0x7fb578142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57fd82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb578153210 0x7fb578150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57fd82640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb578156620 0x7fb5781e2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57fd82640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb578156ab0 con 0x7fb57806fb20 2024-04-16T15:40:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57fd82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb578070dd0 con 0x7fb578153210 2024-04-16T15:40:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57fd82640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb57806e160 con 0x7fb578156620 2024-04-16T15:40:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb57806fb20 0x7fb578142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb57806fb20 0x7fb578142e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49688/0 (socket says 172.21.15.71:49688) 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 -- 172.21.15.71:0/2936918963 learned_addr learned my addr 172.21.15.71:0/2936918963 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57d31d640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb578153210 0x7fb578150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57e31f640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb578156620 0x7fb5781e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 -- 172.21.15.71:0/2936918963 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb578156620 msgr2=0x7fb5781e2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb578156620 0x7fb5781e2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 -- 172.21.15.71:0/2936918963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb578153210 msgr2=0x7fb578150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb578153210 0x7fb578150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 -- 172.21.15.71:0/2936918963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5781ef6f0 con 0x7fb57806fb20 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.796+0000 7fb57db1e640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb57806fb20 0x7fb578142e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb568000f80 tx=0x7fb568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.797+0000 7fb5767fc640 1 -- 172.21.15.71:0/2936918963 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568043020 con 0x7fb57806fb20 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.797+0000 7fb5767fc640 1 -- 172.21.15.71:0/2936918963 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56803e450 con 0x7fb57806fb20 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.797+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5781ef9d0 con 0x7fb57806fb20 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.797+0000 7fb5767fc640 1 -- 172.21.15.71:0/2936918963 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56803e840 con 0x7fb57806fb20 2024-04-16T15:40:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.797+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57814c6c0 con 0x7fb57806fb20 2024-04-16T15:40:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.798+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb578068be0 con 0x7fb57806fb20 2024-04-16T15:40:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.798+0000 7fb5767fc640 1 -- 172.21.15.71:0/2936918963 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb568046020 con 0x7fb57806fb20 2024-04-16T15:40:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.798+0000 7fb5767fc640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.798+0000 7fb5767fc640 1 -- 172.21.15.71:0/2936918963 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb5680ae6b0 con 0x7fb57806fb20 2024-04-16T15:40:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.799+0000 7fb57d31d640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.799+0000 7fb57d31d640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb56406c8f0 0x7fb56406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5780c8360 tx=0x7fb56c009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.803+0000 7fb5767fc640 1 -- 172.21.15.71:0/2936918963 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb56804d030 con 0x7fb57806fb20 2024-04-16T15:40:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.977+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb5780c2620 con 0x7fb57806fb20 2024-04-16T15:40:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.977+0000 7fb5767fc640 1 -- 172.21.15.71:0/2936918963 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fb56807c150 con 0x7fb57806fb20 2024-04-16T15:40:36.976 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.979+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb56406c8f0 msgr2=0x7fb56406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.979+0000 7fb57fd82640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb56406c8f0 0x7fb56406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb5780c8360 tx=0x7fb56c009060 comp rx=0 tx=0).stop 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.979+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb57806fb20 msgr2=0x7fb578142e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.979+0000 7fb57fd82640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb57806fb20 0x7fb578142e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb568000f80 tx=0x7fb568004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.980+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 shutdown_connections 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.980+0000 7fb57fd82640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb56406c8f0 0x7fb56406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.980+0000 7fb57fd82640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb578156620 0x7fb5781e2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.980+0000 7fb57fd82640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb578153210 0x7fb578150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.980+0000 7fb57fd82640 1 --2- 172.21.15.71:0/2936918963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb57806fb20 0x7fb578142e20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.980+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 >> 172.21.15.71:0/2936918963 conn(0x7fb578072aa0 msgr2=0x7fb5780c3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.980+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 shutdown_connections 2024-04-16T15:40:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:36.980+0000 7fb57fd82640 1 -- 172.21.15.71:0/2936918963 wait complete. 2024-04-16T15:40:36.989 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:40:37.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.096+0000 7f80170a5640 1 Processor -- start 2024-04-16T15:40:37.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.096+0000 7f80170a5640 1 -- start start 2024-04-16T15:40:37.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.096+0000 7f80170a5640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f8010151090 0x7f8010071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.096+0000 7f80170a5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8010151470 0x7f80100c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.096+0000 7f80170a5640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80100c5710 con 0x7f80100c3cb0 2024-04-16T15:40:37.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.096+0000 7f80170a5640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f8010154ab0 con 0x7f8010151090 2024-04-16T15:40:37.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.096+0000 7f80170a5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f80100c54e0 con 0x7f8010151470 2024-04-16T15:40:37.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f8014e41640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f8010151090 0x7f8010071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f800ffff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f80100c3cb0 0x7f80100c78a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56102/0 (socket says 172.21.15.71:56102) 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f800ffff640 1 -- 172.21.15.71:0/2072407206 learned_addr learned my addr 172.21.15.71:0/2072407206 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f8015642640 1 --2- 172.21.15.71:0/2072407206 >> v2:172.21.15.122:3300/0 conn(0x7f8010151470 0x7f80100c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f800f7fe640 1 -- 172.21.15.71:0/2072407206 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359313956 0 0) 0x7f80100c5710 con 0x7f80100c3cb0 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f800f7fe640 1 -- 172.21.15.71:0/2072407206 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ffc003520 con 0x7f80100c3cb0 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f8014e41640 1 -- 172.21.15.71:0/2072407206 >> v2:172.21.15.122:3300/0 conn(0x7f8010151470 msgr2=0x7f80100c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f8014e41640 1 --2- 172.21.15.71:0/2072407206 >> v2:172.21.15.122:3300/0 conn(0x7f8010151470 0x7f80100c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f8014e41640 1 -- 172.21.15.71:0/2072407206 >> v1:172.21.15.71:6789/0 conn(0x7f80100c3cb0 legacy=0x7f80100c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f8014e41640 1 -- 172.21.15.71:0/2072407206 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80101544f0 con 0x7f8010151090 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.097+0000 7f8015642640 1 --2- 172.21.15.71:0/2072407206 >> v2:172.21.15.122:3300/0 conn(0x7f8010151470 0x7f80100c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f8014e41640 1 --2- 172.21.15.71:0/2072407206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151090 0x7f8010071bc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f80100730f0 tx=0x7f800802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34c93b503bc1d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f800f7fe640 1 -- 172.21.15.71:0/2072407206 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008039070 con 0x7f8010151090 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f800f7fe640 1 -- 172.21.15.71:0/2072407206 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800802cbf0 con 0x7f8010151090 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f800f7fe640 1 -- 172.21.15.71:0/2072407206 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008034440 con 0x7f8010151090 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f80170a5640 1 -- 172.21.15.71:0/2072407206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151090 msgr2=0x7f8010071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f80170a5640 1 --2- 172.21.15.71:0/2072407206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151090 0x7f8010071bc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f80100730f0 tx=0x7f800802be40 comp rx=0 tx=0).stop 2024-04-16T15:40:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f80170a5640 1 -- 172.21.15.71:0/2072407206 shutdown_connections 2024-04-16T15:40:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f80170a5640 1 --2- 172.21.15.71:0/2072407206 >> v2:172.21.15.122:3300/0 conn(0x7f8010151470 0x7f80100c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f80170a5640 1 --2- 172.21.15.71:0/2072407206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151090 0x7f8010071bc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f80170a5640 1 -- 172.21.15.71:0/2072407206 >> 172.21.15.71:0/2072407206 conn(0x7f80100c5db0 msgr2=0x7f80100c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f80170a5640 1 -- 172.21.15.71:0/2072407206 shutdown_connections 2024-04-16T15:40:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.098+0000 7f80170a5640 1 -- 172.21.15.71:0/2072407206 wait complete. 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.099+0000 7f80170a5640 1 Processor -- start 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.100+0000 7f80170a5640 1 -- start start 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.100+0000 7f80170a5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80100c3cb0 0x7f80101518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.100+0000 7f80170a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8010151090 0x7f8010149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.100+0000 7f80170a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151470 0x7f80101531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.100+0000 7f80170a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f80100c84f0 con 0x7f8010151470 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.100+0000 7f80170a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80100c59b0 con 0x7f8010151090 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.100+0000 7f80170a5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8010072100 con 0x7f80100c3cb0 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.100+0000 7f8015642640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151470 0x7f80101531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8015642640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151470 0x7f80101531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49710/0 (socket says 172.21.15.71:49710) 2024-04-16T15:40:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8015642640 1 -- 172.21.15.71:0/2882186409 learned_addr learned my addr 172.21.15.71:0/2882186409 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f800ffff640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8010151090 0x7f8010149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8014e41640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80100c3cb0 0x7f80101518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8015642640 1 -- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80100c3cb0 msgr2=0x7f80101518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8015642640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80100c3cb0 0x7f80101518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8015642640 1 -- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8010151090 msgr2=0x7f8010149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8015642640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8010151090 0x7f8010149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8015642640 1 -- 172.21.15.71:0/2882186409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80101576a0 con 0x7f8010151470 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f800ffff640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8010151090 0x7f8010149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f8015642640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151470 0x7f80101531a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800e810 tx=0x7f7ff800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f800d7fa640 1 -- 172.21.15.71:0/2882186409 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8017070 con 0x7f8010151470 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f800d7fa640 1 -- 172.21.15.71:0/2882186409 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff800c6d0 con 0x7f8010151470 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80100c2c30 con 0x7f8010151470 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f800d7fa640 1 -- 172.21.15.71:0/2882186409 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800f040 con 0x7f8010151470 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.101+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8010149e40 con 0x7f8010151470 2024-04-16T15:40:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.102+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8010068be0 con 0x7f8010151470 2024-04-16T15:40:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.103+0000 7f800d7fa640 1 -- 172.21.15.71:0/2882186409 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ff801a070 con 0x7f8010151470 2024-04-16T15:40:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.103+0000 7f800d7fa640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ff006c8f0 0x7f7ff006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.103+0000 7f800d7fa640 1 -- 172.21.15.71:0/2882186409 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7ff808b9f0 con 0x7f8010151470 2024-04-16T15:40:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.103+0000 7f8014e41640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ff006c8f0 0x7f7ff006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.104+0000 7f8014e41640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ff006c8f0 0x7f7ff006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8008036000 tx=0x7f8008030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.107+0000 7f800d7fa640 1 -- 172.21.15.71:0/2882186409 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ff8059b70 con 0x7f8010151470 2024-04-16T15:40:37.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.281+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f80100a9ef0 con 0x7f8010151470 2024-04-16T15:40:37.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.281+0000 7f800d7fa640 1 -- 172.21.15.71:0/2882186409 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f7ff8059470 con 0x7f8010151470 2024-04-16T15:40:37.280 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:40:37.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.283+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ff006c8f0 msgr2=0x7f7ff006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.283+0000 7f80170a5640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ff006c8f0 0x7f7ff006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8008036000 tx=0x7f8008030000 comp rx=0 tx=0).stop 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.283+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151470 msgr2=0x7f80101531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.283+0000 7f80170a5640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151470 0x7f80101531a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800e810 tx=0x7f7ff800ecd0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.284+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 shutdown_connections 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.284+0000 7f80170a5640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ff006c8f0 0x7f7ff006edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.284+0000 7f80170a5640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8010151470 0x7f80101531a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.284+0000 7f80170a5640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8010151090 0x7f8010149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.284+0000 7f80170a5640 1 --2- 172.21.15.71:0/2882186409 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80100c3cb0 0x7f80101518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.284+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 >> 172.21.15.71:0/2882186409 conn(0x7f80100c5db0 msgr2=0x7f80100c6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.284+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 shutdown_connections 2024-04-16T15:40:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.284+0000 7f80170a5640 1 -- 172.21.15.71:0/2882186409 wait complete. 2024-04-16T15:40:37.293 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:40:37.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.400+0000 7f5356f1b640 1 Processor -- start 2024-04-16T15:40:37.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.400+0000 7f5356f1b640 1 -- start start 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5356f1b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f53500aa9c0 0x7f535006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5356f1b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f535006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5356f1b640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53500ab030 con 0x7f5350153210 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5356f1b640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f53501535f0 con 0x7f53500aa9c0 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5356f1b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f53500c1c10 con 0x7f535006df90 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5347fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f53500aa9c0 0x7f535006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5354cb7640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f5350153210 0x7f5350156620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:56106/0 (socket says 172.21.15.71:56106) 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5354cb7640 1 -- 172.21.15.71:0/2112416965 learned_addr learned my addr 172.21.15.71:0/2112416965 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f53554b8640 1 --2- 172.21.15.71:0/2112416965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f535006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f53477fe640 1 -- 172.21.15.71:0/2112416965 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1256176111 0 0) 0x7f53500ab030 con 0x7f5350153210 2024-04-16T15:40:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f53477fe640 1 -- 172.21.15.71:0/2112416965 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5338003520 con 0x7f5350153210 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5347fff640 1 -- 172.21.15.71:0/2112416965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 msgr2=0x7f535006fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5347fff640 1 --2- 172.21.15.71:0/2112416965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f535006fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5347fff640 1 -- 172.21.15.71:0/2112416965 >> v1:172.21.15.71:6789/0 conn(0x7f5350153210 legacy=0x7f5350156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f5347fff640 1 -- 172.21.15.71:0/2112416965 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53500c1ef0 con 0x7f53500aa9c0 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.401+0000 7f53554b8640 1 --2- 172.21.15.71:0/2112416965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f535006fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.402+0000 7f5347fff640 1 --2- 172.21.15.71:0/2112416965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53500aa9c0 0x7f535006da50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f53400097a0 tx=0x7f534002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b2edc181668a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.402+0000 7f53477fe640 1 -- 172.21.15.71:0/2112416965 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53400315a0 con 0x7f53500aa9c0 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.402+0000 7f53477fe640 1 -- 172.21.15.71:0/2112416965 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5340037070 con 0x7f53500aa9c0 2024-04-16T15:40:37.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.402+0000 7f53477fe640 1 -- 172.21.15.71:0/2112416965 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340031960 con 0x7f53500aa9c0 2024-04-16T15:40:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.403+0000 7f5356f1b640 1 -- 172.21.15.71:0/2112416965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53500aa9c0 msgr2=0x7f535006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.403+0000 7f5356f1b640 1 --2- 172.21.15.71:0/2112416965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53500aa9c0 0x7f535006da50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f53400097a0 tx=0x7f534002b840 comp rx=0 tx=0).stop 2024-04-16T15:40:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.403+0000 7f5356f1b640 1 -- 172.21.15.71:0/2112416965 shutdown_connections 2024-04-16T15:40:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.403+0000 7f5356f1b640 1 --2- 172.21.15.71:0/2112416965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f535006fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.403+0000 7f5356f1b640 1 --2- 172.21.15.71:0/2112416965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53500aa9c0 0x7f535006da50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.403+0000 7f5356f1b640 1 -- 172.21.15.71:0/2112416965 >> 172.21.15.71:0/2112416965 conn(0x7f5350072aa0 msgr2=0x7f5350072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.403+0000 7f5356f1b640 1 -- 172.21.15.71:0/2112416965 shutdown_connections 2024-04-16T15:40:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.403+0000 7f5356f1b640 1 -- 172.21.15.71:0/2112416965 wait complete. 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.404+0000 7f5356f1b640 1 Processor -- start 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.404+0000 7f5356f1b640 1 -- start start 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5356f1b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f53501508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5356f1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53500aa9c0 0x7f535014e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5356f1b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5350153210 0x7f535014ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5356f1b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53500712f0 con 0x7f5350153210 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5356f1b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5350070060 con 0x7f53500aa9c0 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5356f1b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f535014a640 con 0x7f535006df90 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5350153210 0x7f535014ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5350153210 0x7f535014ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49728/0 (socket says 172.21.15.71:49728) 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 -- 172.21.15.71:0/1965908105 learned_addr learned my addr 172.21.15.71:0/1965908105 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5354cb7640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f53501508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5347fff640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53500aa9c0 0x7f535014e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 -- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 msgr2=0x7f53501508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f53501508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 -- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53500aa9c0 msgr2=0x7f535014e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53500aa9c0 0x7f535014e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 -- 172.21.15.71:0/1965908105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5350152a40 con 0x7f5350153210 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f5347fff640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53500aa9c0 0x7f535014e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.405+0000 7f53554b8640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5350153210 0x7f535014ca80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f534800cf20 tx=0x7f534800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.406+0000 7f53457fa640 1 -- 172.21.15.71:0/1965908105 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534801a070 con 0x7f5350153210 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.406+0000 7f53457fa640 1 -- 172.21.15.71:0/1965908105 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534800d840 con 0x7f5350153210 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.406+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5350152480 con 0x7f5350153210 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.406+0000 7f53457fa640 1 -- 172.21.15.71:0/1965908105 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534800dc30 con 0x7f5350153210 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.406+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53500c7910 con 0x7f5350153210 2024-04-16T15:40:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.407+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5350068be0 con 0x7f5350153210 2024-04-16T15:40:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.407+0000 7f53457fa640 1 -- 172.21.15.71:0/1965908105 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f534801d070 con 0x7f5350153210 2024-04-16T15:40:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.408+0000 7f53457fa640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f533c06c920 0x7f533c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.408+0000 7f53457fa640 1 -- 172.21.15.71:0/1965908105 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f534808c5e0 con 0x7f5350153210 2024-04-16T15:40:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.408+0000 7f5354cb7640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f533c06c920 0x7f533c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.408+0000 7f5354cb7640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f533c06c920 0x7f533c06ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f534c005880 tx=0x7f534c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.412+0000 7f53457fa640 1 -- 172.21.15.71:0/1965908105 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f534805a140 con 0x7f5350153210 2024-04-16T15:40:37.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.587+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f53500c2620 con 0x7f5350153210 2024-04-16T15:40:37.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.588+0000 7f53457fa640 1 -- 172.21.15.71:0/1965908105 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f534805e440 con 0x7f5350153210 2024-04-16T15:40:37.587 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:37.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f533c06c920 msgr2=0x7f533c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f533c06c920 0x7f533c06ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f534c005880 tx=0x7f534c009040 comp rx=0 tx=0).stop 2024-04-16T15:40:37.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5350153210 msgr2=0x7f535014ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5350153210 0x7f535014ca80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f534800cf20 tx=0x7f534800a7d0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 shutdown_connections 2024-04-16T15:40:37.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f533c06c920 0x7f533c06ede0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5350153210 0x7f535014ca80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53500aa9c0 0x7f535014e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 --2- 172.21.15.71:0/1965908105 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f535006df90 0x7f53501508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 >> 172.21.15.71:0/1965908105 conn(0x7f5350072aa0 msgr2=0x7f5350072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:37.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 shutdown_connections 2024-04-16T15:40:37.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.590+0000 7f5356f1b640 1 -- 172.21.15.71:0/1965908105 wait complete. 2024-04-16T15:40:37.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T15:40:37.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.706+0000 7f1ce8b15640 1 Processor -- start 2024-04-16T15:40:37.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.706+0000 7f1ce8b15640 1 -- start start 2024-04-16T15:40:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce8b15640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1ce40c56b0 0x7f1ce4071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce8b15640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1ce4159720 0x7f1ce4159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce8b15640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce4071fe0 con 0x7f1ce4155c10 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce8b15640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce4159290 con 0x7f1ce4159720 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce8b15640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce415a090 con 0x7f1ce40c56b0 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce2d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1ce4159720 0x7f1ce4159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce2d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1ce4159720 0x7f1ce4159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46598/0 (socket says 172.21.15.71:46598) 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce2d9d640 1 -- 172.21.15.71:0/3905559819 learned_addr learned my addr 172.21.15.71:0/3905559819 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.707+0000 7f1ce259c640 1 --2- 172.21.15.71:0/3905559819 >> v2:172.21.15.122:3300/0 conn(0x7f1ce40c56b0 0x7f1ce4071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.708+0000 7f1ce2d9d640 1 -- 172.21.15.71:0/3905559819 >> v2:172.21.15.122:3300/0 conn(0x7f1ce40c56b0 msgr2=0x7f1ce4071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.708+0000 7f1ce2d9d640 1 --2- 172.21.15.71:0/3905559819 >> v2:172.21.15.122:3300/0 conn(0x7f1ce40c56b0 0x7f1ce4071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.708+0000 7f1ce2d9d640 1 -- 172.21.15.71:0/3905559819 >> v1:172.21.15.122:6789/0 conn(0x7f1ce4155c10 legacy=0x7f1ce40c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.708+0000 7f1ce2d9d640 1 -- 172.21.15.71:0/3905559819 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce4156140 con 0x7f1ce4159720 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.708+0000 7f1ce2d9d640 1 --2- 172.21.15.71:0/3905559819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce4159b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4070dd0 tx=0x7f1cdc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfcd0a635dd583e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.708+0000 7f1ce159a640 1 -- 172.21.15.71:0/3905559819 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc034070 con 0x7f1ce4159720 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce159a640 1 -- 172.21.15.71:0/3905559819 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cdc030860 con 0x7f1ce4159720 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce159a640 1 -- 172.21.15.71:0/3905559819 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc030c30 con 0x7f1ce4159720 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce8b15640 1 -- 172.21.15.71:0/3905559819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 msgr2=0x7f1ce4159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/3905559819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce4159b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4070dd0 tx=0x7f1cdc02b9a0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce8b15640 1 -- 172.21.15.71:0/3905559819 shutdown_connections 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/3905559819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce4159b50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/3905559819 >> v2:172.21.15.122:3300/0 conn(0x7f1ce40c56b0 0x7f1ce4071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce8b15640 1 -- 172.21.15.71:0/3905559819 >> 172.21.15.71:0/3905559819 conn(0x7f1ce40ab440 msgr2=0x7f1ce40ab850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:37.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce8b15640 1 -- 172.21.15.71:0/3905559819 shutdown_connections 2024-04-16T15:40:37.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.709+0000 7f1ce8b15640 1 -- 172.21.15.71:0/3905559819 wait complete. 2024-04-16T15:40:37.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.710+0000 7f1ce8b15640 1 Processor -- start 2024-04-16T15:40:37.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.710+0000 7f1ce8b15640 1 -- start start 2024-04-16T15:40:37.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce8b15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce40c56b0 0x7f1ce4070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce8b15640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1ce4155c10 0x7f1ce414ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce8b15640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce414b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce8b15640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41569e0 con 0x7f1ce4159720 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce8b15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4156700 con 0x7f1ce40c56b0 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce8b15640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce40c5a90 con 0x7f1ce4155c10 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce414b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce414b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46606/0 (socket says 172.21.15.71:46606) 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce2d9d640 1 -- 172.21.15.71:0/1785455340 learned_addr learned my addr 172.21.15.71:0/1785455340 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce1d9b640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1ce4155c10 0x7f1ce414ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.711+0000 7f1ce259c640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce40c56b0 0x7f1ce4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce2d9d640 1 -- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1ce4155c10 msgr2=0x7f1ce414ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce2d9d640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1ce4155c10 0x7f1ce414ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce2d9d640 1 -- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce40c56b0 msgr2=0x7f1ce4070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce2d9d640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce40c56b0 0x7f1ce4070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce2d9d640 1 -- 172.21.15.71:0/1785455340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce41eca50 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce259c640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce40c56b0 0x7f1ce4070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce1d9b640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1ce4155c10 0x7f1ce414ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce2d9d640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce414b090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009960 tx=0x7f1cdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1cd2ffd640 1 -- 172.21.15.71:0/1785455340 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc034040 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1cd2ffd640 1 -- 172.21.15.71:0/1785455340 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cdc039d90 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1cd2ffd640 1 -- 172.21.15.71:0/1785455340 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc043610 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce414e940 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.712+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce40706d0 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.713+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce4068c10 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.713+0000 7f1cd2ffd640 1 -- 172.21.15.71:0/1785455340 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cdc037070 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.714+0000 7f1cd2ffd640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1cc406c8f0 0x7f1cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.714+0000 7f1cd2ffd640 1 -- 172.21.15.71:0/1785455340 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1cdc0ade30 con 0x7f1ce4159720 2024-04-16T15:40:37.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.714+0000 7f1ce259c640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1cc406c8f0 0x7f1cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:37.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.714+0000 7f1ce259c640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1cc406c8f0 0x7f1cc406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40074c0 tx=0x7f1cd400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:37.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.718+0000 7f1cd2ffd640 1 -- 172.21.15.71:0/1785455340 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1cdc03e070 con 0x7f1ce4159720 2024-04-16T15:40:37.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.909+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1ce4071880 con 0x7f1ce4159720 2024-04-16T15:40:37.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.909+0000 7f1cd2ffd640 1 -- 172.21.15.71:0/1785455340 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f1cdc07bfb0 con 0x7f1ce4159720 2024-04-16T15:40:37.908 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1cc406c8f0 msgr2=0x7f1cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1cc406c8f0 0x7f1cc406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40074c0 tx=0x7f1cd400b040 comp rx=0 tx=0).stop 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 msgr2=0x7f1ce414b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce414b090 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009960 tx=0x7f1cdc0046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 shutdown_connections 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1cc406c8f0 0x7f1cc406edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ce4159720 0x7f1ce414b090 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1ce4155c10 0x7f1ce414ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 --2- 172.21.15.71:0/1785455340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ce40c56b0 0x7f1ce4070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 >> 172.21.15.71:0/1785455340 conn(0x7f1ce40ab440 msgr2=0x7f1ce41535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 shutdown_connections 2024-04-16T15:40:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:37.911+0000 7f1ce8b15640 1 -- 172.21.15.71:0/1785455340 wait complete. 2024-04-16T15:40:37.921 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2024-04-16T15:40:37.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:37.921 DEBUG:teuthology.orchestra.run.smithi071:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-16T15:40:37.950 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[45297]: fuse finished with error 0 and tester_r 0 2024-04-16T15:40:37.951 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:40:37.983 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:40:37.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:37.983 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-16T15:40:38.049 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:38.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.192+0000 7fcff91d6640 1 Processor -- start 2024-04-16T15:40:38.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.192+0000 7fcff91d6640 1 -- start start 2024-04-16T15:40:38.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff91d6640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcff4155dc0 0x7fcff40c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff91d6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcff406fac0 0x7fcff41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff91d6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff40ab290 0x7fcff41527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff91d6640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fcff4073f20 con 0x7fcff4155dc0 2024-04-16T15:40:38.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff91d6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fcff4156ab0 con 0x7fcff406fac0 2024-04-16T15:40:38.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff91d6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcff40c27c0 con 0x7fcff40ab290 2024-04-16T15:40:38.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff2d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcff4155dc0 0x7fcff40c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff2d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcff4155dc0 0x7fcff40c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46620/0 (socket says 172.21.15.71:46620) 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff2d9d640 1 -- 172.21.15.71:0/3365527324 learned_addr learned my addr 172.21.15.71:0/3365527324 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff359e640 1 --2- 172.21.15.71:0/3365527324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff40ab290 0x7fcff41527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff259c640 1 --2- 172.21.15.71:0/3365527324 >> v2:172.21.15.122:3300/0 conn(0x7fcff406fac0 0x7fcff41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff2d9d640 1 -- 172.21.15.71:0/3365527324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff40ab290 msgr2=0x7fcff41527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff2d9d640 1 --2- 172.21.15.71:0/3365527324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff40ab290 0x7fcff41527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff2d9d640 1 -- 172.21.15.71:0/3365527324 >> v2:172.21.15.122:3300/0 conn(0x7fcff406fac0 msgr2=0x7fcff41561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff2d9d640 1 --2- 172.21.15.71:0/3365527324 >> v2:172.21.15.122:3300/0 conn(0x7fcff406fac0 0x7fcff41561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.193+0000 7fcff2d9d640 1 -- 172.21.15.71:0/3365527324 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff40c2aa0 con 0x7fcff4155dc0 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.194+0000 7fcff2d9d640 1 --2- 172.21.15.71:0/3365527324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff4155dc0 0x7fcff40c32a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcff406d2d0 tx=0x7fcfe402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72333d783b462bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.194+0000 7fcff1d9b640 1 -- 172.21.15.71:0/3365527324 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe402ec80 con 0x7fcff4155dc0 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.194+0000 7fcff1d9b640 1 -- 172.21.15.71:0/3365527324 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe402ee20 con 0x7fcff4155dc0 2024-04-16T15:40:38.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.194+0000 7fcff1d9b640 1 -- 172.21.15.71:0/3365527324 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4036610 con 0x7fcff4155dc0 2024-04-16T15:40:38.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 -- 172.21.15.71:0/3365527324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff4155dc0 msgr2=0x7fcff40c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 --2- 172.21.15.71:0/3365527324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff4155dc0 0x7fcff40c32a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcff406d2d0 tx=0x7fcfe402e0e0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 -- 172.21.15.71:0/3365527324 shutdown_connections 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 --2- 172.21.15.71:0/3365527324 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff40ab290 0x7fcff41527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 --2- 172.21.15.71:0/3365527324 >> v2:172.21.15.122:3300/0 conn(0x7fcff406fac0 0x7fcff41561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 --2- 172.21.15.71:0/3365527324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff4155dc0 0x7fcff40c32a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 -- 172.21.15.71:0/3365527324 >> 172.21.15.71:0/3365527324 conn(0x7fcff40c3e10 msgr2=0x7fcff40c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 -- 172.21.15.71:0/3365527324 shutdown_connections 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.195+0000 7fcff91d6640 1 -- 172.21.15.71:0/3365527324 wait complete. 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.196+0000 7fcff91d6640 1 Processor -- start 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.196+0000 7fcff91d6640 1 -- start start 2024-04-16T15:40:38.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff91d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff40ab290 0x7fcff40c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff91d6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff4155dc0 0x7fcff4141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff91d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcff4072c70 0x7fcff41eab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff91d6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcff41566e0 con 0x7fcff40ab290 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff91d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcff40c18b0 con 0x7fcff4072c70 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff91d6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcff4156860 con 0x7fcff4155dc0 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff40ab290 0x7fcff40c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff40ab290 0x7fcff40c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46628/0 (socket says 172.21.15.71:46628) 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff2d9d640 1 -- 172.21.15.71:0/1493681056 learned_addr learned my addr 172.21.15.71:0/1493681056 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff259c640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff4155dc0 0x7fcff4141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff359e640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcff4072c70 0x7fcff41eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff2d9d640 1 -- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff4155dc0 msgr2=0x7fcff4141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff2d9d640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff4155dc0 0x7fcff4141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff2d9d640 1 -- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcff4072c70 msgr2=0x7fcff41eab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff2d9d640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcff4072c70 0x7fcff41eab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff2d9d640 1 -- 172.21.15.71:0/1493681056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff40724b0 con 0x7fcff40ab290 2024-04-16T15:40:38.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff359e640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcff4072c70 0x7fcff41eab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:38.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.197+0000 7fcff259c640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff4155dc0 0x7fcff4141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:38.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.198+0000 7fcff2d9d640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff40ab290 0x7fcff40c4fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009f40 tx=0x7fcfe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.198+0000 7fcfeb7fe640 1 -- 172.21.15.71:0/1493681056 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4045070 con 0x7fcff40ab290 2024-04-16T15:40:38.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.198+0000 7fcfeb7fe640 1 -- 172.21.15.71:0/1493681056 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe4041660 con 0x7fcff40ab290 2024-04-16T15:40:38.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.198+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff4142310 con 0x7fcff40ab290 2024-04-16T15:40:38.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.198+0000 7fcfeb7fe640 1 -- 172.21.15.71:0/1493681056 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4041a50 con 0x7fcff40ab290 2024-04-16T15:40:38.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.198+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff4154ce0 con 0x7fcff40ab290 2024-04-16T15:40:38.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.199+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff4068d20 con 0x7fcff40ab290 2024-04-16T15:40:38.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.199+0000 7fcfeb7fe640 1 -- 172.21.15.71:0/1493681056 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfe4041bf0 con 0x7fcff40ab290 2024-04-16T15:40:38.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.200+0000 7fcfeb7fe640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfd406c920 0x7fcfd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.200+0000 7fcff259c640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfd406c920 0x7fcfd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.200+0000 7fcfeb7fe640 1 -- 172.21.15.71:0/1493681056 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcfe40b02c0 con 0x7fcff40ab290 2024-04-16T15:40:38.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.200+0000 7fcff259c640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfd406c920 0x7fcfd406ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcff40c2240 tx=0x7fcfdc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.204+0000 7fcfeb7fe640 1 -- 172.21.15.71:0/1493681056 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfe407e440 con 0x7fcff40ab290 2024-04-16T15:40:38.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.381+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcff406bb50 con 0x7fcff40ab290 2024-04-16T15:40:38.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.382+0000 7fcfeb7fe640 1 -- 172.21.15.71:0/1493681056 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7fcfe407dd40 con 0x7fcff40ab290 2024-04-16T15:40:38.382 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:38.382 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:38.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfd406c920 msgr2=0x7fcfd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfd406c920 0x7fcfd406ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcff40c2240 tx=0x7fcfdc008040 comp rx=0 tx=0).stop 2024-04-16T15:40:38.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff40ab290 msgr2=0x7fcff40c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff40ab290 0x7fcff40c4fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009f40 tx=0x7fcfe4004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:38.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 shutdown_connections 2024-04-16T15:40:38.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfd406c920 0x7fcfd406ede0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcff4072c70 0x7fcff41eab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcff4155dc0 0x7fcff4141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 --2- 172.21.15.71:0/1493681056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcff40ab290 0x7fcff40c4fb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 >> 172.21.15.71:0/1493681056 conn(0x7fcff40c3e10 msgr2=0x7fcff41464b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:38.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 shutdown_connections 2024-04-16T15:40:38.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.384+0000 7fcff91d6640 1 -- 172.21.15.71:0/1493681056 wait complete. 2024-04-16T15:40:38.396 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:38.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.498+0000 7f881c304640 1 Processor -- start 2024-04-16T15:40:38.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.498+0000 7f881c304640 1 -- start start 2024-04-16T15:40:38.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881c304640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814151010 0x7f88140c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881c304640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f8814072520 0x7f88140729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881c304640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88140c78b0 con 0x7f8814070810 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881c304640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f8814073f80 con 0x7f8814072520 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881c304640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f88140735e0 con 0x7f8814151010 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881a8a1640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f8814072520 0x7f88140729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881989f640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f8814070810 0x7f881406c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34132/0 (socket says 172.21.15.71:34132) 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881989f640 1 -- 172.21.15.71:0/3485900574 learned_addr learned my addr 172.21.15.71:0/3485900574 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881a0a0640 1 --2- 172.21.15.71:0/3485900574 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814151010 0x7f88140c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881a8a1640 1 -- 172.21.15.71:0/3485900574 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814151010 msgr2=0x7f88140c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881a8a1640 1 --2- 172.21.15.71:0/3485900574 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814151010 0x7f88140c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881a8a1640 1 -- 172.21.15.71:0/3485900574 >> v1:172.21.15.71:6789/0 conn(0x7f8814070810 legacy=0x7f881406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.499+0000 7f881a8a1640 1 -- 172.21.15.71:0/3485900574 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88140738c0 con 0x7f8814072520 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.500+0000 7f881a8a1640 1 --2- 172.21.15.71:0/3485900574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814072520 0x7f88140729b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f880c009d90 tx=0x7f880c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d78c45c7165d0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.500+0000 7f881909e640 1 -- 172.21.15.71:0/3485900574 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c039070 con 0x7f8814072520 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.500+0000 7f881909e640 1 -- 172.21.15.71:0/3485900574 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c02cbc0 con 0x7f8814072520 2024-04-16T15:40:38.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.500+0000 7f881909e640 1 -- 172.21.15.71:0/3485900574 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c034440 con 0x7f8814072520 2024-04-16T15:40:38.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.500+0000 7f881c304640 1 -- 172.21.15.71:0/3485900574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814072520 msgr2=0x7f88140729b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.500+0000 7f881c304640 1 --2- 172.21.15.71:0/3485900574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814072520 0x7f88140729b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f880c009d90 tx=0x7f880c02be10 comp rx=0 tx=0).stop 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.501+0000 7f881c304640 1 -- 172.21.15.71:0/3485900574 shutdown_connections 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.501+0000 7f881c304640 1 --2- 172.21.15.71:0/3485900574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814072520 0x7f88140729b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.501+0000 7f881c304640 1 --2- 172.21.15.71:0/3485900574 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814151010 0x7f88140c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.501+0000 7f881c304640 1 -- 172.21.15.71:0/3485900574 >> 172.21.15.71:0/3485900574 conn(0x7f88140c45d0 msgr2=0x7f88140c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.501+0000 7f881c304640 1 -- 172.21.15.71:0/3485900574 shutdown_connections 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.501+0000 7f881c304640 1 -- 172.21.15.71:0/3485900574 wait complete. 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.502+0000 7f881c304640 1 Processor -- start 2024-04-16T15:40:38.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.502+0000 7f881c304640 1 -- start start 2024-04-16T15:40:38.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.502+0000 7f881c304640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8814070810 0x7f88141ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.502+0000 7f881c304640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814072520 0x7f88141ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.502+0000 7f881c304640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814151010 0x7f88141def30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.502+0000 7f881c304640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f881406d050 con 0x7f8814151010 2024-04-16T15:40:38.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.502+0000 7f881c304640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88140c7570 con 0x7f8814070810 2024-04-16T15:40:38.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.502+0000 7f881c304640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f881406d1d0 con 0x7f8814072520 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814151010 0x7f88141def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814151010 0x7f88141def30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46660/0 (socket says 172.21.15.71:46660) 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 -- 172.21.15.71:0/2293263458 learned_addr learned my addr 172.21.15.71:0/2293263458 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881989f640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814072520 0x7f88141ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a0a0640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8814070810 0x7f88141ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 -- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814072520 msgr2=0x7f88141ed0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814072520 0x7f88141ed0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 -- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8814070810 msgr2=0x7f88141ecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8814070810 0x7f88141ecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 -- 172.21.15.71:0/2293263458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88141dea50 con 0x7f8814151010 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881989f640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814072520 0x7f88141ed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a0a0640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8814070810 0x7f88141ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:38.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881a8a1640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814151010 0x7f88141def30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f880c009eb0 tx=0x7f880c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f8806ffd640 1 -- 172.21.15.71:0/2293263458 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c039050 con 0x7f8814151010 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f8806ffd640 1 -- 172.21.15.71:0/2293263458 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c045070 con 0x7f8814151010 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88141ed9d0 con 0x7f8814151010 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f8806ffd640 1 -- 172.21.15.71:0/2293263458 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c03e830 con 0x7f8814151010 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.503+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88141e0d00 con 0x7f8814151010 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.504+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8814068d20 con 0x7f8814151010 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.505+0000 7f8806ffd640 1 -- 172.21.15.71:0/2293263458 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f880c03e9d0 con 0x7f8814151010 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.505+0000 7f8806ffd640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f87f806c8f0 0x7f87f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.505+0000 7f8806ffd640 1 -- 172.21.15.71:0/2293263458 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f880c0aeb80 con 0x7f8814151010 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.505+0000 7f881a0a0640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f87f806c8f0 0x7f87f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.505+0000 7f881a0a0640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f87f806c8f0 0x7f87f806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8808006fd0 tx=0x7f8808008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.509+0000 7f8806ffd640 1 -- 172.21.15.71:0/2293263458 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f880c032e20 con 0x7f8814151010 2024-04-16T15:40:38.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.685+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88141e0f40 con 0x7f8814151010 2024-04-16T15:40:38.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.686+0000 7f8806ffd640 1 -- 172.21.15.71:0/2293263458 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f880c07ce20 con 0x7f8814151010 2024-04-16T15:40:38.686 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:38.687 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:38.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f87f806c8f0 msgr2=0x7f87f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f87f806c8f0 0x7f87f806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8808006fd0 tx=0x7f8808008040 comp rx=0 tx=0).stop 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814151010 msgr2=0x7f88141def30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814151010 0x7f88141def30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f880c009eb0 tx=0x7f880c004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 shutdown_connections 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f87f806c8f0 0x7f87f806edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8814151010 0x7f88141def30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8814072520 0x7f88141ed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 --2- 172.21.15.71:0/2293263458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8814070810 0x7f88141ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.688+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 >> 172.21.15.71:0/2293263458 conn(0x7f88140c45d0 msgr2=0x7f881414f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.689+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 shutdown_connections 2024-04-16T15:40:38.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.689+0000 7f881c304640 1 -- 172.21.15.71:0/2293263458 wait complete. 2024-04-16T15:40:38.700 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-16T15:40:38.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.807+0000 7f44545db640 1 Processor -- start 2024-04-16T15:40:38.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.807+0000 7f44545db640 1 -- start start 2024-04-16T15:40:38.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.808+0000 7f44545db640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f444c153310 0x7f444c159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.808+0000 7f44545db640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.808+0000 7f44545db640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f444c0c2a40 con 0x7f444c0c8f90 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.808+0000 7f44545db640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f444c0c1bc0 con 0x7f444c153310 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.808+0000 7f44545db640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f444c0c4c40 con 0x7f444c0c2e70 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452377640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f444c153310 0x7f444c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452377640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f444c153310 0x7f444c159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:50918/0 (socket says 172.21.15.71:50918) 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4451b76640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f444c0c8f90 0x7f444c0c2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:60688/0 (socket says 172.21.15.71:60688) 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452377640 1 -- 172.21.15.71:0/1312168601 learned_addr learned my addr 172.21.15.71:0/1312168601 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452b78640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452377640 1 -- 172.21.15.71:0/1312168601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 msgr2=0x7f444c0c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452377640 1 --2- 172.21.15.71:0/1312168601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c0c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452377640 1 -- 172.21.15.71:0/1312168601 >> v1:172.21.15.122:6789/0 conn(0x7f444c0c8f90 legacy=0x7f444c0c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452377640 1 -- 172.21.15.71:0/1312168601 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c156880 con 0x7f444c153310 2024-04-16T15:40:38.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.809+0000 7f4452b78640 1 --2- 172.21.15.71:0/1312168601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.810+0000 7f4452377640 1 --2- 172.21.15.71:0/1312168601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c153310 0x7f444c159330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4448009870 tx=0x7f444802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38a04814636fb69 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.810+0000 7f4451375640 1 -- 172.21.15.71:0/1312168601 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44480317c0 con 0x7f444c153310 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.810+0000 7f4451375640 1 -- 172.21.15.71:0/1312168601 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4448037070 con 0x7f444c153310 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.810+0000 7f4451375640 1 -- 172.21.15.71:0/1312168601 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448031b80 con 0x7f444c153310 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.810+0000 7f44545db640 1 -- 172.21.15.71:0/1312168601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c153310 msgr2=0x7f444c159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.810+0000 7f44545db640 1 --2- 172.21.15.71:0/1312168601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c153310 0x7f444c159330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4448009870 tx=0x7f444802b860 comp rx=0 tx=0).stop 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.810+0000 7f44545db640 1 -- 172.21.15.71:0/1312168601 shutdown_connections 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.811+0000 7f44545db640 1 --2- 172.21.15.71:0/1312168601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.811+0000 7f44545db640 1 --2- 172.21.15.71:0/1312168601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c153310 0x7f444c159330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.811+0000 7f44545db640 1 -- 172.21.15.71:0/1312168601 >> 172.21.15.71:0/1312168601 conn(0x7f444c0c8470 msgr2=0x7f444c0c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.811+0000 7f44545db640 1 -- 172.21.15.71:0/1312168601 shutdown_connections 2024-04-16T15:40:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.811+0000 7f44545db640 1 -- 172.21.15.71:0/1312168601 wait complete. 2024-04-16T15:40:38.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f44545db640 1 Processor -- start 2024-04-16T15:40:38.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f44545db640 1 -- start start 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f44545db640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f44545db640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f444c153310 0x7f444c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f44545db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c06dba0 0x7f444c154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f44545db640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0c47d0 con 0x7f444c153310 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f44545db640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f444c1536f0 con 0x7f444c06dba0 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f44545db640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f444c0c4950 con 0x7f444c0c2e70 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f4451b76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f444c153310 0x7f444c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f4451b76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f444c153310 0x7f444c1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46670/0 (socket says 172.21.15.71:46670) 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.812+0000 7f4451b76640 1 -- 172.21.15.71:0/571219983 learned_addr learned my addr 172.21.15.71:0/571219983 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4452377640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4452b78640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c06dba0 0x7f444c154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4451b76640 1 -- 172.21.15.71:0/571219983 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 msgr2=0x7f444c157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4451b76640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4451b76640 1 -- 172.21.15.71:0/571219983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c06dba0 msgr2=0x7f444c154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4451b76640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c06dba0 0x7f444c154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4451b76640 1 -- 172.21.15.71:0/571219983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c0c7630 con 0x7f444c153310 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4451b76640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f444c153310 0x7f444c1583b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f443c00c430 tx=0x7f443c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4442ffd640 1 -- 172.21.15.71:0/571219983 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0074e0 con 0x7f444c153310 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4442ffd640 1 -- 172.21.15.71:0/571219983 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443c007680 con 0x7f444c153310 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c1543c0 con 0x7f444c153310 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f4442ffd640 1 -- 172.21.15.71:0/571219983 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c004a00 con 0x7f444c153310 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.813+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c0c6f60 con 0x7f444c153310 2024-04-16T15:40:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.814+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c068be0 con 0x7f444c153310 2024-04-16T15:40:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.815+0000 7f4442ffd640 1 -- 172.21.15.71:0/571219983 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f443c01a020 con 0x7f444c153310 2024-04-16T15:40:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.815+0000 7f4442ffd640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f443006c8f0 0x7f443006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.815+0000 7f4442ffd640 1 -- 172.21.15.71:0/571219983 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f443c08b6b0 con 0x7f444c153310 2024-04-16T15:40:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.815+0000 7f4452377640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f443006c8f0 0x7f443006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.815+0000 7f4452377640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f443006c8f0 0x7f443006edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4448036000 tx=0x7f444803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:38.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:38.819+0000 7f4442ffd640 1 -- 172.21.15.71:0/571219983 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f443c0597b0 con 0x7f444c153310 2024-04-16T15:40:39.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.035+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f444c0c19a0 con 0x7f444c153310 2024-04-16T15:40:39.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.770+0000 7f4442ffd640 1 -- 172.21.15.71:0/571219983 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f443c0590b0 con 0x7f444c153310 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f443006c8f0 msgr2=0x7f443006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f443006c8f0 0x7f443006edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4448036000 tx=0x7f444803d040 comp rx=0 tx=0).stop 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f444c153310 msgr2=0x7f444c1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f444c153310 0x7f444c1583b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f443c00c430 tx=0x7f443c00c8f0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 shutdown_connections 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f443006c8f0 0x7f443006edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f444c06dba0 0x7f444c154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f444c153310 0x7f444c1583b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 --2- 172.21.15.71:0/571219983 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f444c0c2e70 0x7f444c157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 >> 172.21.15.71:0/571219983 conn(0x7f444c0c8470 msgr2=0x7f444c0737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:39.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 shutdown_connections 2024-04-16T15:40:39.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.773+0000 7f44545db640 1 -- 172.21.15.71:0/571219983 wait complete. 2024-04-16T15:40:39.786 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-16T15:40:39.786 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-16T15:40:39.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.894+0000 7fdad9260640 1 Processor -- start 2024-04-16T15:40:39.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.894+0000 7fdad9260640 1 -- start start 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad9260640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdad4153450 0x7fdad40c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad9260640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad40c99d0 0x7fdad4155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad9260640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad406e820 con 0x7fdad40c94c0 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad9260640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fdad4159fb0 con 0x7fdad4153450 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad9260640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdad40c8150 con 0x7fdad40c99d0 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad2d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdad4153450 0x7fdad40c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad359e640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fdad40c94c0 0x7fdad41562f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34148/0 (socket says 172.21.15.71:34148) 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad359e640 1 -- 172.21.15.71:0/1047581451 learned_addr learned my addr 172.21.15.71:0/1047581451 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdaca59c640 1 --2- 172.21.15.71:0/1047581451 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad40c99d0 0x7fdad4155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad2d9d640 1 -- 172.21.15.71:0/1047581451 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad40c99d0 msgr2=0x7fdad4155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad2d9d640 1 --2- 172.21.15.71:0/1047581451 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad40c99d0 0x7fdad4155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad2d9d640 1 -- 172.21.15.71:0/1047581451 >> v1:172.21.15.71:6789/0 conn(0x7fdad40c94c0 legacy=0x7fdad41562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:39.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad2d9d640 1 -- 172.21.15.71:0/1047581451 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40c8430 con 0x7fdad4153450 2024-04-16T15:40:39.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.895+0000 7fdad259c640 1 -- 172.21.15.71:0/1047581451 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 467510860 0 0) 0x7fdad406e820 con 0x7fdad40c94c0 2024-04-16T15:40:39.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.896+0000 7fdad2d9d640 1 --2- 172.21.15.71:0/1047581451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad4153450 0x7fdad40c5f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdac40097a0 tx=0x7fdac402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7e0e9829e9a501 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:39.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.896+0000 7fdad259c640 1 -- 172.21.15.71:0/1047581451 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4038070 con 0x7fdad4153450 2024-04-16T15:40:39.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.896+0000 7fdad259c640 1 -- 172.21.15.71:0/1047581451 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac4031700 con 0x7fdad4153450 2024-04-16T15:40:39.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.896+0000 7fdad259c640 1 -- 172.21.15.71:0/1047581451 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4031ad0 con 0x7fdad4153450 2024-04-16T15:40:39.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.897+0000 7fdad9260640 1 -- 172.21.15.71:0/1047581451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad4153450 msgr2=0x7fdad40c5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:39.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.897+0000 7fdad9260640 1 --2- 172.21.15.71:0/1047581451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad4153450 0x7fdad40c5f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdac40097a0 tx=0x7fdac402b7a0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.897+0000 7fdad9260640 1 -- 172.21.15.71:0/1047581451 shutdown_connections 2024-04-16T15:40:39.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.897+0000 7fdad9260640 1 --2- 172.21.15.71:0/1047581451 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad40c99d0 0x7fdad4155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.897+0000 7fdad9260640 1 --2- 172.21.15.71:0/1047581451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad4153450 0x7fdad40c5f40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.897+0000 7fdad9260640 1 -- 172.21.15.71:0/1047581451 >> 172.21.15.71:0/1047581451 conn(0x7fdad40ad2f0 msgr2=0x7fdad40ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:39.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.897+0000 7fdad9260640 1 -- 172.21.15.71:0/1047581451 shutdown_connections 2024-04-16T15:40:39.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.897+0000 7fdad9260640 1 -- 172.21.15.71:0/1047581451 wait complete. 2024-04-16T15:40:39.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.898+0000 7fdad9260640 1 Processor -- start 2024-04-16T15:40:39.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.898+0000 7fdad9260640 1 -- start start 2024-04-16T15:40:39.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad9260640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad40c94c0 0x7fdad4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:39.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad9260640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad40c99d0 0x7fdad41e8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:39.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad9260640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad4153450 0x7fdad41e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:39.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad9260640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40c6480 con 0x7fdad40c99d0 2024-04-16T15:40:39.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad9260640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdad4159c60 con 0x7fdad40c94c0 2024-04-16T15:40:39.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad9260640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdad40c6600 con 0x7fdad4153450 2024-04-16T15:40:39.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad40c99d0 0x7fdad41e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:39.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad40c99d0 0x7fdad41e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46688/0 (socket says 172.21.15.71:46688) 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 -- 172.21.15.71:0/590050090 learned_addr learned my addr 172.21.15.71:0/590050090 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad2d9d640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad40c94c0 0x7fdad4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad359e640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad4153450 0x7fdad41e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 -- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad4153450 msgr2=0x7fdad41e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad4153450 0x7fdad41e8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 -- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad40c94c0 msgr2=0x7fdad4151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad40c94c0 0x7fdad4151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 -- 172.21.15.71:0/590050090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad414d840 con 0x7fdad40c99d0 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad2d9d640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad40c94c0 0x7fdad4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdad359e640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad4153450 0x7fdad41e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.899+0000 7fdaca59c640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad40c99d0 0x7fdad41e8580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00d620 tx=0x7fdabc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:39.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.900+0000 7fdacbfff640 1 -- 172.21.15.71:0/590050090 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00b620 con 0x7fdad40c99d0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.900+0000 7fdacbfff640 1 -- 172.21.15.71:0/590050090 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdabc019070 con 0x7fdad40c99d0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.900+0000 7fdacbfff640 1 -- 172.21.15.71:0/590050090 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00ba00 con 0x7fdad40c99d0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.900+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41eeb40 con 0x7fdad40c99d0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.900+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41ef100 con 0x7fdad40c99d0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.901+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad4074060 con 0x7fdad40c99d0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.901+0000 7fdacbfff640 1 -- 172.21.15.71:0/590050090 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdabc00bba0 con 0x7fdad40c99d0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.901+0000 7fdacbfff640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdab406c920 0x7fdab406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.902+0000 7fdacbfff640 1 -- 172.21.15.71:0/590050090 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdabc08c290 con 0x7fdad40c99d0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.902+0000 7fdad2d9d640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdab406c920 0x7fdab406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:39.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.902+0000 7fdad2d9d640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdab406c920 0x7fdab406ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdac4035000 tx=0x7fdac4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:39.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:39.906+0000 7fdacbfff640 1 -- 172.21.15.71:0/590050090 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdabc05a380 con 0x7fdad40c99d0 2024-04-16T15:40:40.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.118+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdad414e2b0 con 0x7fdad40c99d0 2024-04-16T15:40:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.772+0000 7fdacbfff640 1 -- 172.21.15.71:0/590050090 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fdabc059c80 con 0x7fdad40c99d0 2024-04-16T15:40:40.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdab406c920 msgr2=0x7fdab406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:40.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdab406c920 0x7fdab406ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdac4035000 tx=0x7fdac4008030 comp rx=0 tx=0).stop 2024-04-16T15:40:40.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad40c99d0 msgr2=0x7fdad41e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad40c99d0 0x7fdad41e8580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00d620 tx=0x7fdabc00dae0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 shutdown_connections 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdab406c920 0x7fdab406ede0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdad4153450 0x7fdad41e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdad40c99d0 0x7fdad41e8580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 --2- 172.21.15.71:0/590050090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad40c94c0 0x7fdad4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 >> 172.21.15.71:0/590050090 conn(0x7fdad40ad2f0 msgr2=0x7fdad40ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 shutdown_connections 2024-04-16T15:40:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.774+0000 7fdad9260640 1 -- 172.21.15.71:0/590050090 wait complete. 2024-04-16T15:40:40.787 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-16T15:40:40.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:40:40.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.891+0000 7f9fa8574640 1 Processor -- start 2024-04-16T15:40:40.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.891+0000 7f9fa8574640 1 -- start start 2024-04-16T15:40:40.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.892+0000 7f9fa8574640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9fa0153450 0x7f9fa00c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:40.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.892+0000 7f9fa8574640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9fa00c99d0 0x7f9fa0155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:40.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.892+0000 7f9fa8574640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa006e820 con 0x7f9fa00c94c0 2024-04-16T15:40:40.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.892+0000 7f9fa8574640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa0159fb0 con 0x7f9fa0153450 2024-04-16T15:40:40.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.892+0000 7f9fa8574640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa00c8150 con 0x7f9fa00c99d0 2024-04-16T15:40:40.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.892+0000 7f9fa6310640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9fa0153450 0x7f9fa00c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:40.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.892+0000 7f9fa6310640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9fa0153450 0x7f9fa00c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46704/0 (socket says 172.21.15.71:46704) 2024-04-16T15:40:40.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.892+0000 7f9fa6310640 1 -- 172.21.15.71:0/99431261 learned_addr learned my addr 172.21.15.71:0/99431261 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:40.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.893+0000 7f9fa6310640 1 -- 172.21.15.71:0/99431261 >> v2:172.21.15.122:3300/0 conn(0x7f9fa00c99d0 msgr2=0x7f9fa0155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:40.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.893+0000 7f9fa6310640 1 --2- 172.21.15.71:0/99431261 >> v2:172.21.15.122:3300/0 conn(0x7f9fa00c99d0 0x7f9fa0155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.893+0000 7f9fa6310640 1 -- 172.21.15.71:0/99431261 >> v1:172.21.15.122:6789/0 conn(0x7f9fa00c94c0 legacy=0x7f9fa01562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:40.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.893+0000 7f9fa6310640 1 -- 172.21.15.71:0/99431261 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa00c8430 con 0x7f9fa0153450 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.893+0000 7f9fa6310640 1 --2- 172.21.15.71:0/99431261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa0153450 0x7f9fa00c5f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fa008cf60 tx=0x7f9f9002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3781cf4579120d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.893+0000 7f9fa530e640 1 -- 172.21.15.71:0/99431261 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9002c960 con 0x7f9fa0153450 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.893+0000 7f9fa530e640 1 -- 172.21.15.71:0/99431261 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9002cb00 con 0x7f9fa0153450 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.893+0000 7f9fa530e640 1 -- 172.21.15.71:0/99431261 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900343f0 con 0x7f9fa0153450 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.894+0000 7f9fa8574640 1 -- 172.21.15.71:0/99431261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa0153450 msgr2=0x7f9fa00c5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.894+0000 7f9fa8574640 1 --2- 172.21.15.71:0/99431261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa0153450 0x7f9fa00c5f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9fa008cf60 tx=0x7f9f9002bdc0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.894+0000 7f9fa8574640 1 -- 172.21.15.71:0/99431261 shutdown_connections 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.894+0000 7f9fa8574640 1 --2- 172.21.15.71:0/99431261 >> v2:172.21.15.122:3300/0 conn(0x7f9fa00c99d0 0x7f9fa0155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.894+0000 7f9fa8574640 1 --2- 172.21.15.71:0/99431261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa0153450 0x7f9fa00c5f40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.894+0000 7f9fa8574640 1 -- 172.21.15.71:0/99431261 >> 172.21.15.71:0/99431261 conn(0x7f9fa00ad2f0 msgr2=0x7f9fa00ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.894+0000 7f9fa8574640 1 -- 172.21.15.71:0/99431261 shutdown_connections 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.894+0000 7f9fa8574640 1 -- 172.21.15.71:0/99431261 wait complete. 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.895+0000 7f9fa8574640 1 Processor -- start 2024-04-16T15:40:40.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.895+0000 7f9fa8574640 1 -- start start 2024-04-16T15:40:40.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa8574640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa00c94c0 0x7f9fa01542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa8574640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa00c99d0 0x7f9fa0150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa8574640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9fa0153450 0x7f9fa0157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa8574640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa00c6480 con 0x7f9fa00c99d0 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa8574640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0159c60 con 0x7f9fa00c94c0 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa8574640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa00c6600 con 0x7f9fa0153450 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa00c99d0 0x7f9fa0150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa00c99d0 0x7f9fa0150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46712/0 (socket says 172.21.15.71:46712) 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 -- 172.21.15.71:0/3048091424 learned_addr learned my addr 172.21.15.71:0/3048091424 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa6310640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa00c94c0 0x7f9fa01542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa6b11640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9fa0153450 0x7f9fa0157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 -- 172.21.15.71:0/3048091424 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9fa0153450 msgr2=0x7f9fa0157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:40.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9fa0153450 0x7f9fa0157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 -- 172.21.15.71:0/3048091424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa00c94c0 msgr2=0x7f9fa01542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa00c94c0 0x7f9fa01542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 -- 172.21.15.71:0/3048091424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa0154800 con 0x7f9fa00c99d0 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.896+0000 7f9fa5b0f640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa00c99d0 0x7f9fa0150600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400b440 tx=0x7f9f9400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.897+0000 7f9f9effd640 1 -- 172.21.15.71:0/3048091424 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94015020 con 0x7f9fa00c99d0 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.897+0000 7f9f9effd640 1 -- 172.21.15.71:0/3048091424 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f940118b0 con 0x7f9fa00c99d0 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.897+0000 7f9f9effd640 1 -- 172.21.15.71:0/3048091424 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94011ca0 con 0x7f9fa00c99d0 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.897+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa00ab2e0 con 0x7f9fa00c99d0 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.897+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa01588a0 con 0x7f9fa00c99d0 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.898+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa0068d20 con 0x7f9fa00c99d0 2024-04-16T15:40:40.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.898+0000 7f9f9effd640 1 -- 172.21.15.71:0/3048091424 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f94018020 con 0x7f9fa00c99d0 2024-04-16T15:40:40.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.899+0000 7f9f9effd640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:40.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.899+0000 7f9fa6310640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:40.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.899+0000 7f9f9effd640 1 -- 172.21.15.71:0/3048091424 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9f9408be90 con 0x7f9fa00c99d0 2024-04-16T15:40:40.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.899+0000 7f9fa6310640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f90035000 tx=0x7f9f90030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:40.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:40.903+0000 7f9f9effd640 1 -- 172.21.15.71:0/3048091424 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f9401f020 con 0x7f9fa00c99d0 2024-04-16T15:40:41.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.114+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fa0074950 con 0x7f9fa00c99d0 2024-04-16T15:40:41.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.115+0000 7f9f9effd640 1 -- 172.21.15.71:0/3048091424 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f9f9400d040 con 0x7f9fa00c99d0 2024-04-16T15:40:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:41.115 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:40.120873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:40:41.115 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.116+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9f8c06c8f0 msgr2=0x7f9f8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.116+0000 7f9fa8574640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f90035000 tx=0x7f9f90030000 comp rx=0 tx=0).stop 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.116+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa00c99d0 msgr2=0x7f9fa0150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.116+0000 7f9fa8574640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa00c99d0 0x7f9fa0150600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400b440 tx=0x7f9f9400b900 comp rx=0 tx=0).stop 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.117+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 shutdown_connections 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.117+0000 7f9fa8574640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9f8c06c8f0 0x7f9f8c06edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.117+0000 7f9fa8574640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9fa0153450 0x7f9fa0157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.117+0000 7f9fa8574640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fa00c99d0 0x7f9fa0150600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.117+0000 7f9fa8574640 1 --2- 172.21.15.71:0/3048091424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa00c94c0 0x7f9fa01542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.117+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 >> 172.21.15.71:0/3048091424 conn(0x7f9fa00ad2f0 msgr2=0x7f9fa00cada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.117+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 shutdown_connections 2024-04-16T15:40:41.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.117+0000 7f9fa8574640 1 -- 172.21.15.71:0/3048091424 wait complete. 2024-04-16T15:40:41.128 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:41.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.230+0000 7ff35c4b1640 1 Processor -- start 2024-04-16T15:40:41.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.230+0000 7ff35c4b1640 1 -- start start 2024-04-16T15:40:41.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.231+0000 7ff35c4b1640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff354159520 0x7ff354159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.231+0000 7ff35c4b1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff35406df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.231+0000 7ff35c4b1640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3540ae4b0 con 0x7ff35406e470 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.231+0000 7ff35c4b1640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff3540c9540 con 0x7ff354159520 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.231+0000 7ff35c4b1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff3540c6940 con 0x7ff3540c78d0 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.231+0000 7ff35aa4e640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7ff35406e470 0x7ff3540ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34154/0 (socket says 172.21.15.71:34154) 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.231+0000 7ff35aa4e640 1 -- 172.21.15.71:0/489227160 learned_addr learned my addr 172.21.15.71:0/489227160 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff35a24d640 1 --2- 172.21.15.71:0/489227160 >> v2:172.21.15.122:3300/0 conn(0x7ff354159520 0x7ff354159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff359a4c640 1 --2- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff35406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff35924b640 1 -- 172.21.15.71:0/489227160 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838695167 0 0) 0x7ff3540ae4b0 con 0x7ff35406e470 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff35924b640 1 -- 172.21.15.71:0/489227160 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff340003520 con 0x7ff35406e470 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff35a24d640 1 -- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 msgr2=0x7ff35406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff35a24d640 1 --2- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff35406df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff35a24d640 1 -- 172.21.15.71:0/489227160 >> v1:172.21.15.71:6789/0 conn(0x7ff35406e470 legacy=0x7ff3540ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff35924b640 1 -- 172.21.15.71:0/489227160 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 210552665 0 0) 0x7ff340003520 con 0x7ff35406e470 2024-04-16T15:40:41.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff35a24d640 1 -- 172.21.15.71:0/489227160 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35415a030 con 0x7ff354159520 2024-04-16T15:40:41.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.232+0000 7ff359a4c640 1 --2- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff35406df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:41.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.233+0000 7ff35a24d640 1 --2- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 0x7ff354159900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff3540c96c0 tx=0x7ff35002bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=103ccd1bce88f067 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.233+0000 7ff35924b640 1 -- 172.21.15.71:0/489227160 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350039070 con 0x7ff354159520 2024-04-16T15:40:41.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.233+0000 7ff35924b640 1 -- 172.21.15.71:0/489227160 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff35002c780 con 0x7ff354159520 2024-04-16T15:40:41.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.233+0000 7ff35924b640 1 -- 172.21.15.71:0/489227160 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c001670 con 0x7ff354159520 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.233+0000 7ff35c4b1640 1 -- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 msgr2=0x7ff354159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.233+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 0x7ff354159900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff3540c96c0 tx=0x7ff35002bc50 comp rx=0 tx=0).stop 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.234+0000 7ff35c4b1640 1 -- 172.21.15.71:0/489227160 shutdown_connections 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.234+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff35406df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.234+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/489227160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 0x7ff354159900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.234+0000 7ff35c4b1640 1 -- 172.21.15.71:0/489227160 >> 172.21.15.71:0/489227160 conn(0x7ff3540cb820 msgr2=0x7ff3540cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.234+0000 7ff35c4b1640 1 -- 172.21.15.71:0/489227160 shutdown_connections 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.234+0000 7ff35c4b1640 1 -- 172.21.15.71:0/489227160 wait complete. 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35c4b1640 1 Processor -- start 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35c4b1640 1 -- start start 2024-04-16T15:40:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35c4b1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff35406e470 0x7ff3541584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35c4b1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff3541ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35c4b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 0x7ff3541eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35c4b1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3540c9d50 con 0x7ff35406e470 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35c4b1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff3540c9a70 con 0x7ff354159520 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35c4b1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff3540b0cf0 con 0x7ff3540c78d0 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35a24d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff35406e470 0x7ff3541584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35a24d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff35406e470 0x7ff3541584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46720/0 (socket says 172.21.15.71:46720) 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.235+0000 7ff35a24d640 1 -- 172.21.15.71:0/933439898 learned_addr learned my addr 172.21.15.71:0/933439898 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff359a4c640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff3541ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35aa4e640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 0x7ff3541eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35a24d640 1 -- 172.21.15.71:0/933439898 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 msgr2=0x7ff3541ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35a24d640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff3541ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35a24d640 1 -- 172.21.15.71:0/933439898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 msgr2=0x7ff3541eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35a24d640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 0x7ff3541eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35a24d640 1 -- 172.21.15.71:0/933439898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3541ef360 con 0x7ff35406e470 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35a24d640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff35406e470 0x7ff3541584a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff350009e80 tx=0x7ff350004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff34affd640 1 -- 172.21.15.71:0/933439898 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350039050 con 0x7ff35406e470 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff34affd640 1 -- 172.21.15.71:0/933439898 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff350045070 con 0x7ff35406e470 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3540ab560 con 0x7ff35406e470 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff34affd640 1 -- 172.21.15.71:0/933439898 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350040450 con 0x7ff35406e470 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.236+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3541566c0 con 0x7ff35406e470 2024-04-16T15:40:41.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.237+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff354074060 con 0x7ff35406e470 2024-04-16T15:40:41.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.238+0000 7ff34affd640 1 -- 172.21.15.71:0/933439898 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3500405f0 con 0x7ff35406e470 2024-04-16T15:40:41.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.238+0000 7ff34affd640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff33806c8f0 0x7ff33806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.238+0000 7ff34affd640 1 -- 172.21.15.71:0/933439898 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff3500ae7e0 con 0x7ff35406e470 2024-04-16T15:40:41.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.238+0000 7ff359a4c640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff33806c8f0 0x7ff33806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.239+0000 7ff359a4c640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff33806c8f0 0x7ff33806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff344009560 tx=0x7ff344008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.243+0000 7ff34affd640 1 -- 172.21.15.71:0/933439898 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3541ef360 con 0x7ff35406e470 2024-04-16T15:40:41.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.422+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3540bebc0 con 0x7ff35406e470 2024-04-16T15:40:41.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.423+0000 7ff34affd640 1 -- 172.21.15.71:0/933439898 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7ff35007c840 con 0x7ff35406e470 2024-04-16T15:40:41.423 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:41.423 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff33806c8f0 msgr2=0x7ff33806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff33806c8f0 0x7ff33806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff344009560 tx=0x7ff344008040 comp rx=0 tx=0).stop 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff35406e470 msgr2=0x7ff3541584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff35406e470 0x7ff3541584a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff350009e80 tx=0x7ff350004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 shutdown_connections 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff33806c8f0 0x7ff33806edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff354159520 0x7ff3541eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3540c78d0 0x7ff3541ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 --2- 172.21.15.71:0/933439898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff35406e470 0x7ff3541584a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 >> 172.21.15.71:0/933439898 conn(0x7ff3540cb820 msgr2=0x7ff3540ca170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 shutdown_connections 2024-04-16T15:40:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.425+0000 7ff35c4b1640 1 -- 172.21.15.71:0/933439898 wait complete. 2024-04-16T15:40:41.436 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-16T15:40:41.436 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:40:41.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.538+0000 7f0582b1c640 1 Processor -- start 2024-04-16T15:40:41.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.538+0000 7f0582b1c640 1 -- start start 2024-04-16T15:40:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f0582b1c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 0x7f057c0acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f0582b1c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f057c06ff00 0x7f057c070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f0582b1c640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057c073f70 con 0x7f057c153140 2024-04-16T15:40:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f0582b1c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f057c06de20 con 0x7f057c06ff00 2024-04-16T15:40:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f0582b1c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f057c06db60 con 0x7f057c06e040 2024-04-16T15:40:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f05808b8640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f057c153140 0x7f057c156570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34160/0 (socket says 172.21.15.71:34160) 2024-04-16T15:40:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f05808b8640 1 -- 172.21.15.71:0/3323493254 learned_addr learned my addr 172.21.15.71:0/3323493254 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:41.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f05810b9640 1 --2- 172.21.15.71:0/3323493254 >> v2:172.21.15.122:3300/0 conn(0x7f057c06ff00 0x7f057c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057bfff640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 0x7f057c0acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 542720931 0 0) 0x7f057c073f70 con 0x7f057c153140 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f056c003520 con 0x7f057c153140 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3010217514 0 0) 0x7f056c003520 con 0x7f057c153140 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 msgr2=0x7f057c0acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 0x7f057c0acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 >> v2:172.21.15.122:3300/0 conn(0x7f057c06ff00 msgr2=0x7f057c070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 --2- 172.21.15.71:0/3323493254 >> v2:172.21.15.122:3300/0 conn(0x7f057c06ff00 0x7f057c070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c159280 con 0x7f057c153140 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f0570001670 con 0x7f057c153140 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057bfff640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 0x7f057c0acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:41.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 >> v1:172.21.15.71:6789/0 conn(0x7f057c153140 legacy=0x7f057c156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f056c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 0x7f056c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c004d40 0x7f056c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f057c06de20 con 0x7f056c0039b0 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f057c06db60 con 0x7f056c004d40 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f056c00ba10 con 0x7f057c06e040 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f05808b8640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f056c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.539+0000 7f057bfff640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 0x7f056c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.540+0000 7f05810b9640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c004d40 0x7f056c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.540+0000 7f05808b8640 1 -- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 msgr2=0x7f056c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.540+0000 7f05808b8640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c06e040 0x7f056c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.540+0000 7f05808b8640 1 -- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c004d40 msgr2=0x7f056c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.540+0000 7f05808b8640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f056c004d40 0x7f056c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.540+0000 7f05808b8640 1 -- 172.21.15.71:0/3323493254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c003520 con 0x7f056c0039b0 2024-04-16T15:40:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.540+0000 7f05808b8640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f056c003dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f057000c240 tx=0x7f056c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9cfd653b0eac71 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570007d30 con 0x7f056c0039b0 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570030070 con 0x7f056c0039b0 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f057b7fe640 1 -- 172.21.15.71:0/3323493254 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057002cd60 con 0x7f056c0039b0 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f0582b1c640 1 -- 172.21.15.71:0/3323493254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 msgr2=0x7f056c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f0582b1c640 1 --2- 172.21.15.71:0/3323493254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f056c003dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f057000c240 tx=0x7f056c005830 comp rx=0 tx=0).stop 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f05810b9640 1 -- 172.21.15.71:0/3323493254 reap_dead start 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f0582b1c640 1 -- 172.21.15.71:0/3323493254 shutdown_connections 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f0582b1c640 1 -- 172.21.15.71:0/3323493254 >> 172.21.15.71:0/3323493254 conn(0x7f057c0c8920 msgr2=0x7f057c0c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f0582b1c640 1 -- 172.21.15.71:0/3323493254 shutdown_connections 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.541+0000 7f0582b1c640 1 -- 172.21.15.71:0/3323493254 wait complete. 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.542+0000 7f0582b1c640 1 Processor -- start 2024-04-16T15:40:41.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.542+0000 7f0582b1c640 1 -- start start 2024-04-16T15:40:41.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f0582b1c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f057c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f0582b1c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c155970 0x7f057c1ec240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f0582b1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f057c14ae30 0x7f057c14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f0582b1c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f057c153520 con 0x7f056c0039b0 2024-04-16T15:40:41.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f0582b1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f057c156050 con 0x7f057c14ae30 2024-04-16T15:40:41.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f0582b1c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f057c0736a0 con 0x7f057c155970 2024-04-16T15:40:41.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f05808b8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f057c150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f05808b8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f057c150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46736/0 (socket says 172.21.15.71:46736) 2024-04-16T15:40:41.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f05808b8640 1 -- 172.21.15.71:0/1958514215 learned_addr learned my addr 172.21.15.71:0/1958514215 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f05808b8640 1 -- 172.21.15.71:0/1958514215 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c155970 msgr2=0x7f057c1ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f05808b8640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c155970 0x7f057c1ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f05808b8640 1 -- 172.21.15.71:0/1958514215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f057c14ae30 msgr2=0x7f057c14a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.543+0000 7f05808b8640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f057c14ae30 0x7f057c14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.544+0000 7f05808b8640 1 -- 172.21.15.71:0/1958514215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c152040 con 0x7f056c0039b0 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.544+0000 7f05808b8640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f057c150960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0570001970 tx=0x7f057c159450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.544+0000 7f05797fa640 1 -- 172.21.15.71:0/1958514215 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570001c70 con 0x7f056c0039b0 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.544+0000 7f05797fa640 1 -- 172.21.15.71:0/1958514215 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570004bf0 con 0x7f056c0039b0 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.544+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057c152320 con 0x7f056c0039b0 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.544+0000 7f05797fa640 1 -- 172.21.15.71:0/1958514215 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570007070 con 0x7f056c0039b0 2024-04-16T15:40:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.544+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c1ebd20 con 0x7f056c0039b0 2024-04-16T15:40:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.545+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057c068cf0 con 0x7f056c0039b0 2024-04-16T15:40:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.545+0000 7f05797fa640 1 -- 172.21.15.71:0/1958514215 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0570007210 con 0x7f056c0039b0 2024-04-16T15:40:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.546+0000 7f05797fa640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f056006c8f0 0x7f056006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.546+0000 7f05797fa640 1 -- 172.21.15.71:0/1958514215 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f05700aceb0 con 0x7f056c0039b0 2024-04-16T15:40:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.546+0000 7f057bfff640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f056006c8f0 0x7f056006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.546+0000 7f057bfff640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f056006c8f0 0x7f056006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0568006fa0 tx=0x7f0568006f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.550+0000 7f05797fa640 1 -- 172.21.15.71:0/1958514215 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f057007b030 con 0x7f056c0039b0 2024-04-16T15:40:41.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.721+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f057c073220 con 0x7f056006c8f0 2024-04-16T15:40:41.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.729+0000 7f05797fa640 1 -- 172.21.15.71:0/1958514215 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f057c073220 con 0x7f056006c8f0 2024-04-16T15:40:41.734 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:41.737 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:40:41.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f056006c8f0 msgr2=0x7f056006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f056006c8f0 0x7f056006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0568006fa0 tx=0x7f0568006f10 comp rx=0 tx=0).stop 2024-04-16T15:40:41.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 msgr2=0x7f057c150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f057c150960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0570001970 tx=0x7f057c159450 comp rx=0 tx=0).stop 2024-04-16T15:40:41.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 shutdown_connections 2024-04-16T15:40:41.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f056006c8f0 0x7f056006edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f057c14ae30 0x7f057c14a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f057c155970 0x7f057c1ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 --2- 172.21.15.71:0/1958514215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f056c0039b0 0x7f057c150960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 >> 172.21.15.71:0/1958514215 conn(0x7f057c0c8920 msgr2=0x7f057c14f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:41.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 shutdown_connections 2024-04-16T15:40:41.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.733+0000 7f0582b1c640 1 -- 172.21.15.71:0/1958514215 wait complete. 2024-04-16T15:40:41.744 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-16T15:40:41.698963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219396,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755324,"statfs":{"total":773094113280,"available":772869451776,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002523"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919216+0000","last_change":"2024-04-16T15:40:23.532452+0000","last_active":"2024-04-16T15:40:29.919216+0000","last_peered":"2024-04-16T15:40:29.919216+0000","last_clean":"2024-04-16T15:40:29.919216+0000","last_became_active":"2024-04-16T15:40:23.531709+0000","last_became_peered":"2024-04-16T15:40:23.531709+0000","last_unstale":"2024-04-16T15:40:29.919216+0000","last_undegraded":"2024-04-16T15:40:29.919216+0000","last_fullsized":"2024-04-16T15:40:29.919216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:52.586318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567735+0000","last_change":"2024-04-16T15:40:24.542948+0000","last_active":"2024-04-16T15:40:29.567735+0000","last_peered":"2024-04-16T15:40:29.567735+0000","last_clean":"2024-04-16T15:40:29.567735+0000","last_became_active":"2024-04-16T15:40:24.537898+0000","last_became_peered":"2024-04-16T15:40:24.537898+0000","last_unstale":"2024-04-16T15:40:29.567735+0000","last_undegraded":"2024-04-16T15:40:29.567735+0000","last_fullsized":"2024-04-16T15:40:29.567735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:18.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566318+0000","last_change":"2024-04-16T15:40:28.807073+0000","last_active":"2024-04-16T15:40:29.566318+0000","last_peered":"2024-04-16T15:40:29.566318+0000","last_clean":"2024-04-16T15:40:29.566318+0000","last_became_active":"2024-04-16T15:40:28.806446+0000","last_became_peered":"2024-04-16T15:40:28.806446+0000","last_unstale":"2024-04-16T15:40:29.566318+0000","last_undegraded":"2024-04-16T15:40:29.566318+0000","last_fullsized":"2024-04-16T15:40:29.566318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:54.213728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567940+0000","last_change":"2024-04-16T15:40:23.526932+0000","last_active":"2024-04-16T15:40:29.567940+0000","last_peered":"2024-04-16T15:40:29.567940+0000","last_clean":"2024-04-16T15:40:29.567940+0000","last_became_active":"2024-04-16T15:40:23.526663+0000","last_became_peered":"2024-04-16T15:40:23.526663+0000","last_unstale":"2024-04-16T15:40:29.567940+0000","last_undegraded":"2024-04-16T15:40:29.567940+0000","last_fullsized":"2024-04-16T15:40:29.567940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:52.586318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919103+0000","last_change":"2024-04-16T15:40:24.541524+0000","last_active":"2024-04-16T15:40:29.919103+0000","last_peered":"2024-04-16T15:40:29.919103+0000","last_clean":"2024-04-16T15:40:29.919103+0000","last_became_active":"2024-04-16T15:40:24.540243+0000","last_became_peered":"2024-04-16T15:40:24.540243+0000","last_unstale":"2024-04-16T15:40:29.919103+0000","last_undegraded":"2024-04-16T15:40:29.919103+0000","last_fullsized":"2024-04-16T15:40:29.919103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919188+0000","last_change":"2024-04-16T15:40:28.569991+0000","last_active":"2024-04-16T15:40:29.919188+0000","last_peered":"2024-04-16T15:40:29.919188+0000","last_clean":"2024-04-16T15:40:29.919188+0000","last_became_active":"2024-04-16T15:40:28.569596+0000","last_became_peered":"2024-04-16T15:40:28.569596+0000","last_unstale":"2024-04-16T15:40:29.919188+0000","last_undegraded":"2024-04-16T15:40:29.919188+0000","last_fullsized":"2024-04-16T15:40:29.919188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:21.999059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921685+0000","last_change":"2024-04-16T15:40:23.535294+0000","last_active":"2024-04-16T15:40:29.921685+0000","last_peered":"2024-04-16T15:40:29.921685+0000","last_clean":"2024-04-16T15:40:29.921685+0000","last_became_active":"2024-04-16T15:40:23.534958+0000","last_became_peered":"2024-04-16T15:40:23.534958+0000","last_unstale":"2024-04-16T15:40:29.921685+0000","last_undegraded":"2024-04-16T15:40:29.921685+0000","last_fullsized":"2024-04-16T15:40:29.921685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:59.914744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919928+0000","last_change":"2024-04-16T15:40:24.544556+0000","last_active":"2024-04-16T15:40:29.919928+0000","last_peered":"2024-04-16T15:40:29.919928+0000","last_clean":"2024-04-16T15:40:29.919928+0000","last_became_active":"2024-04-16T15:40:24.544072+0000","last_became_peered":"2024-04-16T15:40:24.544072+0000","last_unstale":"2024-04-16T15:40:29.919928+0000","last_undegraded":"2024-04-16T15:40:29.919928+0000","last_fullsized":"2024-04-16T15:40:29.919928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:25.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920842+0000","last_change":"2024-04-16T15:40:28.570724+0000","last_active":"2024-04-16T15:40:29.920842+0000","last_peered":"2024-04-16T15:40:29.920842+0000","last_clean":"2024-04-16T15:40:29.920842+0000","last_became_active":"2024-04-16T15:40:28.570338+0000","last_became_peered":"2024-04-16T15:40:28.570338+0000","last_unstale":"2024-04-16T15:40:29.920842+0000","last_undegraded":"2024-04-16T15:40:29.920842+0000","last_fullsized":"2024-04-16T15:40:29.920842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:55.765984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920279+0000","last_change":"2024-04-16T15:40:23.536319+0000","last_active":"2024-04-16T15:40:29.920279+0000","last_peered":"2024-04-16T15:40:29.920279+0000","last_clean":"2024-04-16T15:40:29.920279+0000","last_became_active":"2024-04-16T15:40:23.536059+0000","last_became_peered":"2024-04-16T15:40:23.536059+0000","last_unstale":"2024-04-16T15:40:29.920279+0000","last_undegraded":"2024-04-16T15:40:29.920279+0000","last_fullsized":"2024-04-16T15:40:29.920279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:19.974713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920444+0000","last_change":"2024-04-16T15:40:24.543661+0000","last_active":"2024-04-16T15:40:29.920444+0000","last_peered":"2024-04-16T15:40:29.920444+0000","last_clean":"2024-04-16T15:40:29.920444+0000","last_became_active":"2024-04-16T15:40:24.543414+0000","last_became_peered":"2024-04-16T15:40:24.543414+0000","last_unstale":"2024-04-16T15:40:29.920444+0000","last_undegraded":"2024-04-16T15:40:29.920444+0000","last_fullsized":"2024-04-16T15:40:29.920444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:38.664081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919473+0000","last_change":"2024-04-16T15:40:28.566254+0000","last_active":"2024-04-16T15:40:29.919473+0000","last_peered":"2024-04-16T15:40:29.919473+0000","last_clean":"2024-04-16T15:40:29.919473+0000","last_became_active":"2024-04-16T15:40:28.565804+0000","last_became_peered":"2024-04-16T15:40:28.565804+0000","last_unstale":"2024-04-16T15:40:29.919473+0000","last_undegraded":"2024-04-16T15:40:29.919473+0000","last_fullsized":"2024-04-16T15:40:29.919473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:40.008247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566632+0000","last_change":"2024-04-16T15:40:24.539072+0000","last_active":"2024-04-16T15:40:29.566632+0000","last_peered":"2024-04-16T15:40:29.566632+0000","last_clean":"2024-04-16T15:40:29.566632+0000","last_became_active":"2024-04-16T15:40:24.537100+0000","last_became_peered":"2024-04-16T15:40:24.537100+0000","last_unstale":"2024-04-16T15:40:29.566632+0000","last_undegraded":"2024-04-16T15:40:29.566632+0000","last_fullsized":"2024-04-16T15:40:29.566632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920246+0000","last_change":"2024-04-16T15:40:23.537232+0000","last_active":"2024-04-16T15:40:29.920246+0000","last_peered":"2024-04-16T15:40:29.920246+0000","last_clean":"2024-04-16T15:40:29.920246+0000","last_became_active":"2024-04-16T15:40:23.536233+0000","last_became_peered":"2024-04-16T15:40:23.536233+0000","last_unstale":"2024-04-16T15:40:29.920246+0000","last_undegraded":"2024-04-16T15:40:29.920246+0000","last_fullsized":"2024-04-16T15:40:29.920246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:43.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921321+0000","last_change":"2024-04-16T15:40:28.570768+0000","last_active":"2024-04-16T15:40:29.921321+0000","last_peered":"2024-04-16T15:40:29.921321+0000","last_clean":"2024-04-16T15:40:29.921321+0000","last_became_active":"2024-04-16T15:40:28.570416+0000","last_became_peered":"2024-04-16T15:40:28.570416+0000","last_unstale":"2024-04-16T15:40:29.921321+0000","last_undegraded":"2024-04-16T15:40:29.921321+0000","last_fullsized":"2024-04-16T15:40:29.921321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567237+0000","last_change":"2024-04-16T15:40:24.548809+0000","last_active":"2024-04-16T15:40:29.567237+0000","last_peered":"2024-04-16T15:40:29.567237+0000","last_clean":"2024-04-16T15:40:29.567237+0000","last_became_active":"2024-04-16T15:40:24.548463+0000","last_became_peered":"2024-04-16T15:40:24.548463+0000","last_unstale":"2024-04-16T15:40:29.567237+0000","last_undegraded":"2024-04-16T15:40:29.567237+0000","last_fullsized":"2024-04-16T15:40:29.567237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769455+0000","last_change":"2024-04-16T15:40:23.531263+0000","last_active":"2024-04-16T15:40:29.769455+0000","last_peered":"2024-04-16T15:40:29.769455+0000","last_clean":"2024-04-16T15:40:29.769455+0000","last_became_active":"2024-04-16T15:40:23.530998+0000","last_became_peered":"2024-04-16T15:40:23.530998+0000","last_unstale":"2024-04-16T15:40:29.769455+0000","last_undegraded":"2024-04-16T15:40:29.769455+0000","last_fullsized":"2024-04-16T15:40:29.769455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919978+0000","last_change":"2024-04-16T15:40:28.569962+0000","last_active":"2024-04-16T15:40:29.919978+0000","last_peered":"2024-04-16T15:40:29.919978+0000","last_clean":"2024-04-16T15:40:29.919978+0000","last_became_active":"2024-04-16T15:40:28.569572+0000","last_became_peered":"2024-04-16T15:40:28.569572+0000","last_unstale":"2024-04-16T15:40:29.919978+0000","last_undegraded":"2024-04-16T15:40:29.919978+0000","last_fullsized":"2024-04-16T15:40:29.919978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:18.103136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566305+0000","last_change":"2024-04-16T15:40:24.544276+0000","last_active":"2024-04-16T15:40:29.566305+0000","last_peered":"2024-04-16T15:40:29.566305+0000","last_clean":"2024-04-16T15:40:29.566305+0000","last_became_active":"2024-04-16T15:40:24.544035+0000","last_became_peered":"2024-04-16T15:40:24.544035+0000","last_unstale":"2024-04-16T15:40:29.566305+0000","last_undegraded":"2024-04-16T15:40:29.566305+0000","last_fullsized":"2024-04-16T15:40:29.566305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769209+0000","last_change":"2024-04-16T15:40:23.530210+0000","last_active":"2024-04-16T15:40:29.769209+0000","last_peered":"2024-04-16T15:40:29.769209+0000","last_clean":"2024-04-16T15:40:29.769209+0000","last_became_active":"2024-04-16T15:40:23.529955+0000","last_became_peered":"2024-04-16T15:40:23.529955+0000","last_unstale":"2024-04-16T15:40:29.769209+0000","last_undegraded":"2024-04-16T15:40:29.769209+0000","last_fullsized":"2024-04-16T15:40:29.769209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:21.943427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769997+0000","last_change":"2024-04-16T15:40:28.805304+0000","last_active":"2024-04-16T15:40:29.769997+0000","last_peered":"2024-04-16T15:40:29.769997+0000","last_clean":"2024-04-16T15:40:29.769997+0000","last_became_active":"2024-04-16T15:40:28.804926+0000","last_became_peered":"2024-04-16T15:40:28.804926+0000","last_unstale":"2024-04-16T15:40:29.769997+0000","last_undegraded":"2024-04-16T15:40:29.769997+0000","last_fullsized":"2024-04-16T15:40:29.769997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:25.828695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919801+0000","last_change":"2024-04-16T15:40:24.537825+0000","last_active":"2024-04-16T15:40:29.919801+0000","last_peered":"2024-04-16T15:40:29.919801+0000","last_clean":"2024-04-16T15:40:29.919801+0000","last_became_active":"2024-04-16T15:40:24.536478+0000","last_became_peered":"2024-04-16T15:40:24.536478+0000","last_unstale":"2024-04-16T15:40:29.919801+0000","last_undegraded":"2024-04-16T15:40:29.919801+0000","last_fullsized":"2024-04-16T15:40:29.919801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:26.549813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919917+0000","last_change":"2024-04-16T15:40:23.535794+0000","last_active":"2024-04-16T15:40:29.919917+0000","last_peered":"2024-04-16T15:40:29.919917+0000","last_clean":"2024-04-16T15:40:29.919917+0000","last_became_active":"2024-04-16T15:40:23.535373+0000","last_became_peered":"2024-04-16T15:40:23.535373+0000","last_unstale":"2024-04-16T15:40:29.919917+0000","last_undegraded":"2024-04-16T15:40:29.919917+0000","last_fullsized":"2024-04-16T15:40:29.919917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920113+0000","last_change":"2024-04-16T15:40:28.561348+0000","last_active":"2024-04-16T15:40:29.920113+0000","last_peered":"2024-04-16T15:40:29.920113+0000","last_clean":"2024-04-16T15:40:29.920113+0000","last_became_active":"2024-04-16T15:40:28.560833+0000","last_became_peered":"2024-04-16T15:40:28.560833+0000","last_unstale":"2024-04-16T15:40:29.920113+0000","last_undegraded":"2024-04-16T15:40:29.920113+0000","last_fullsized":"2024-04-16T15:40:29.920113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:22.352368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919131+0000","last_change":"2024-04-16T15:40:24.534724+0000","last_active":"2024-04-16T15:40:29.919131+0000","last_peered":"2024-04-16T15:40:29.919131+0000","last_clean":"2024-04-16T15:40:29.919131+0000","last_became_active":"2024-04-16T15:40:24.534469+0000","last_became_peered":"2024-04-16T15:40:24.534469+0000","last_unstale":"2024-04-16T15:40:29.919131+0000","last_undegraded":"2024-04-16T15:40:29.919131+0000","last_fullsized":"2024-04-16T15:40:29.919131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:50:03.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567368+0000","last_change":"2024-04-16T15:40:23.531841+0000","last_active":"2024-04-16T15:40:29.567368+0000","last_peered":"2024-04-16T15:40:29.567368+0000","last_clean":"2024-04-16T15:40:29.567368+0000","last_became_active":"2024-04-16T15:40:23.531565+0000","last_became_peered":"2024-04-16T15:40:23.531565+0000","last_unstale":"2024-04-16T15:40:29.567368+0000","last_undegraded":"2024-04-16T15:40:29.567368+0000","last_fullsized":"2024-04-16T15:40:29.567368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919600+0000","last_change":"2024-04-16T15:40:28.807348+0000","last_active":"2024-04-16T15:40:29.919600+0000","last_peered":"2024-04-16T15:40:29.919600+0000","last_clean":"2024-04-16T15:40:29.919600+0000","last_became_active":"2024-04-16T15:40:28.806984+0000","last_became_peered":"2024-04-16T15:40:28.806984+0000","last_unstale":"2024-04-16T15:40:29.919600+0000","last_undegraded":"2024-04-16T15:40:29.919600+0000","last_fullsized":"2024-04-16T15:40:29.919600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:30.543430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567817+0000","last_change":"2024-04-16T15:40:24.544401+0000","last_active":"2024-04-16T15:40:29.567817+0000","last_peered":"2024-04-16T15:40:29.567817+0000","last_clean":"2024-04-16T15:40:29.567817+0000","last_became_active":"2024-04-16T15:40:24.544156+0000","last_became_peered":"2024-04-16T15:40:24.544156+0000","last_unstale":"2024-04-16T15:40:29.567817+0000","last_undegraded":"2024-04-16T15:40:29.567817+0000","last_fullsized":"2024-04-16T15:40:29.567817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769950+0000","last_change":"2024-04-16T15:40:23.532974+0000","last_active":"2024-04-16T15:40:29.769950+0000","last_peered":"2024-04-16T15:40:29.769950+0000","last_clean":"2024-04-16T15:40:29.769950+0000","last_became_active":"2024-04-16T15:40:23.532692+0000","last_became_peered":"2024-04-16T15:40:23.532692+0000","last_unstale":"2024-04-16T15:40:29.769950+0000","last_undegraded":"2024-04-16T15:40:29.769950+0000","last_fullsized":"2024-04-16T15:40:29.769950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:19.974713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566451+0000","last_change":"2024-04-16T15:40:28.806764+0000","last_active":"2024-04-16T15:40:29.566451+0000","last_peered":"2024-04-16T15:40:29.566451+0000","last_clean":"2024-04-16T15:40:29.566451+0000","last_became_active":"2024-04-16T15:40:28.806437+0000","last_became_peered":"2024-04-16T15:40:28.806437+0000","last_unstale":"2024-04-16T15:40:29.566451+0000","last_undegraded":"2024-04-16T15:40:29.566451+0000","last_fullsized":"2024-04-16T15:40:29.566451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:52:28.865900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769595+0000","last_change":"2024-04-16T15:40:24.536656+0000","last_active":"2024-04-16T15:40:29.769595+0000","last_peered":"2024-04-16T15:40:29.769595+0000","last_clean":"2024-04-16T15:40:29.769595+0000","last_became_active":"2024-04-16T15:40:24.536405+0000","last_became_peered":"2024-04-16T15:40:24.536405+0000","last_unstale":"2024-04-16T15:40:29.769595+0000","last_undegraded":"2024-04-16T15:40:29.769595+0000","last_fullsized":"2024-04-16T15:40:29.769595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921784+0000","last_change":"2024-04-16T15:40:23.536026+0000","last_active":"2024-04-16T15:40:29.921784+0000","last_peered":"2024-04-16T15:40:29.921784+0000","last_clean":"2024-04-16T15:40:29.921784+0000","last_became_active":"2024-04-16T15:40:23.535220+0000","last_became_peered":"2024-04-16T15:40:23.535220+0000","last_unstale":"2024-04-16T15:40:29.921784+0000","last_undegraded":"2024-04-16T15:40:29.921784+0000","last_fullsized":"2024-04-16T15:40:29.921784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:37.570307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918806+0000","last_change":"2024-04-16T15:40:28.804863+0000","last_active":"2024-04-16T15:40:29.918806+0000","last_peered":"2024-04-16T15:40:29.918806+0000","last_clean":"2024-04-16T15:40:29.918806+0000","last_became_active":"2024-04-16T15:40:28.804502+0000","last_became_peered":"2024-04-16T15:40:28.804502+0000","last_unstale":"2024-04-16T15:40:29.918806+0000","last_undegraded":"2024-04-16T15:40:29.918806+0000","last_fullsized":"2024-04-16T15:40:29.918806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920557+0000","last_change":"2024-04-16T15:40:24.551868+0000","last_active":"2024-04-16T15:40:29.920557+0000","last_peered":"2024-04-16T15:40:29.920557+0000","last_clean":"2024-04-16T15:40:29.920557+0000","last_became_active":"2024-04-16T15:40:24.550950+0000","last_became_peered":"2024-04-16T15:40:24.550950+0000","last_unstale":"2024-04-16T15:40:29.920557+0000","last_undegraded":"2024-04-16T15:40:29.920557+0000","last_fullsized":"2024-04-16T15:40:29.920557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:27.826954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769685+0000","last_change":"2024-04-16T15:40:23.533782+0000","last_active":"2024-04-16T15:40:29.769685+0000","last_peered":"2024-04-16T15:40:29.769685+0000","last_clean":"2024-04-16T15:40:29.769685+0000","last_became_active":"2024-04-16T15:40:23.533547+0000","last_became_peered":"2024-04-16T15:40:23.533547+0000","last_unstale":"2024-04-16T15:40:29.769685+0000","last_undegraded":"2024-04-16T15:40:29.769685+0000","last_fullsized":"2024-04-16T15:40:29.769685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769337+0000","last_change":"2024-04-16T15:40:28.570047+0000","last_active":"2024-04-16T15:40:29.769337+0000","last_peered":"2024-04-16T15:40:29.769337+0000","last_clean":"2024-04-16T15:40:29.769337+0000","last_became_active":"2024-04-16T15:40:28.569688+0000","last_became_peered":"2024-04-16T15:40:28.569688+0000","last_unstale":"2024-04-16T15:40:29.769337+0000","last_undegraded":"2024-04-16T15:40:29.769337+0000","last_fullsized":"2024-04-16T15:40:29.769337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:42.690952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921046+0000","last_change":"2024-04-16T15:40:24.549077+0000","last_active":"2024-04-16T15:40:29.921046+0000","last_peered":"2024-04-16T15:40:29.921046+0000","last_clean":"2024-04-16T15:40:29.921046+0000","last_became_active":"2024-04-16T15:40:24.548802+0000","last_became_peered":"2024-04-16T15:40:24.548802+0000","last_unstale":"2024-04-16T15:40:29.921046+0000","last_undegraded":"2024-04-16T15:40:29.921046+0000","last_fullsized":"2024-04-16T15:40:29.921046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:50.142130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920546+0000","last_change":"2024-04-16T15:40:23.528707+0000","last_active":"2024-04-16T15:40:29.920546+0000","last_peered":"2024-04-16T15:40:29.920546+0000","last_clean":"2024-04-16T15:40:29.920546+0000","last_became_active":"2024-04-16T15:40:23.528426+0000","last_became_peered":"2024-04-16T15:40:23.528426+0000","last_unstale":"2024-04-16T15:40:29.920546+0000","last_undegraded":"2024-04-16T15:40:29.920546+0000","last_fullsized":"2024-04-16T15:40:29.920546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769721+0000","last_change":"2024-04-16T15:40:28.561110+0000","last_active":"2024-04-16T15:40:29.769721+0000","last_peered":"2024-04-16T15:40:29.769721+0000","last_clean":"2024-04-16T15:40:29.769721+0000","last_became_active":"2024-04-16T15:40:28.560719+0000","last_became_peered":"2024-04-16T15:40:28.560719+0000","last_unstale":"2024-04-16T15:40:29.769721+0000","last_undegraded":"2024-04-16T15:40:29.769721+0000","last_fullsized":"2024-04-16T15:40:29.769721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:31.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769261+0000","last_change":"2024-04-16T15:40:24.536757+0000","last_active":"2024-04-16T15:40:29.769261+0000","last_peered":"2024-04-16T15:40:29.769261+0000","last_clean":"2024-04-16T15:40:29.769261+0000","last_became_active":"2024-04-16T15:40:24.536520+0000","last_became_peered":"2024-04-16T15:40:24.536520+0000","last_unstale":"2024-04-16T15:40:29.769261+0000","last_undegraded":"2024-04-16T15:40:29.769261+0000","last_fullsized":"2024-04-16T15:40:29.769261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566917+0000","last_change":"2024-04-16T15:40:23.520551+0000","last_active":"2024-04-16T15:40:29.566917+0000","last_peered":"2024-04-16T15:40:29.566917+0000","last_clean":"2024-04-16T15:40:29.566917+0000","last_became_active":"2024-04-16T15:40:23.520300+0000","last_became_peered":"2024-04-16T15:40:23.520300+0000","last_unstale":"2024-04-16T15:40:29.566917+0000","last_undegraded":"2024-04-16T15:40:29.566917+0000","last_fullsized":"2024-04-16T15:40:29.566917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.770097+0000","last_change":"2024-04-16T15:40:28.570553+0000","last_active":"2024-04-16T15:40:29.770097+0000","last_peered":"2024-04-16T15:40:29.770097+0000","last_clean":"2024-04-16T15:40:29.770097+0000","last_became_active":"2024-04-16T15:40:28.569635+0000","last_became_peered":"2024-04-16T15:40:28.569635+0000","last_unstale":"2024-04-16T15:40:29.770097+0000","last_undegraded":"2024-04-16T15:40:29.770097+0000","last_fullsized":"2024-04-16T15:40:29.770097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:28.407124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918519+0000","last_change":"2024-04-16T15:40:24.537795+0000","last_active":"2024-04-16T15:40:29.918519+0000","last_peered":"2024-04-16T15:40:29.918519+0000","last_clean":"2024-04-16T15:40:29.918519+0000","last_became_active":"2024-04-16T15:40:24.537528+0000","last_became_peered":"2024-04-16T15:40:24.537528+0000","last_unstale":"2024-04-16T15:40:29.918519+0000","last_undegraded":"2024-04-16T15:40:29.918519+0000","last_fullsized":"2024-04-16T15:40:29.918519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:38.574731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920406+0000","last_change":"2024-04-16T15:40:23.533872+0000","last_active":"2024-04-16T15:40:29.920406+0000","last_peered":"2024-04-16T15:40:29.920406+0000","last_clean":"2024-04-16T15:40:29.920406+0000","last_became_active":"2024-04-16T15:40:23.533491+0000","last_became_peered":"2024-04-16T15:40:23.533491+0000","last_unstale":"2024-04-16T15:40:29.920406+0000","last_undegraded":"2024-04-16T15:40:29.920406+0000","last_fullsized":"2024-04-16T15:40:29.920406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:21.943427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919078+0000","last_change":"2024-04-16T15:40:28.807602+0000","last_active":"2024-04-16T15:40:29.919078+0000","last_peered":"2024-04-16T15:40:29.919078+0000","last_clean":"2024-04-16T15:40:29.919078+0000","last_became_active":"2024-04-16T15:40:28.807193+0000","last_became_peered":"2024-04-16T15:40:28.807193+0000","last_unstale":"2024-04-16T15:40:29.919078+0000","last_undegraded":"2024-04-16T15:40:29.919078+0000","last_fullsized":"2024-04-16T15:40:29.919078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:35.057588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567071+0000","last_change":"2024-04-16T15:40:24.546548+0000","last_active":"2024-04-16T15:40:29.567071+0000","last_peered":"2024-04-16T15:40:29.567071+0000","last_clean":"2024-04-16T15:40:29.567071+0000","last_became_active":"2024-04-16T15:40:24.546317+0000","last_became_peered":"2024-04-16T15:40:24.546317+0000","last_unstale":"2024-04-16T15:40:29.567071+0000","last_undegraded":"2024-04-16T15:40:29.567071+0000","last_fullsized":"2024-04-16T15:40:29.567071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:14.076265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921124+0000","last_change":"2024-04-16T15:40:23.529866+0000","last_active":"2024-04-16T15:40:29.921124+0000","last_peered":"2024-04-16T15:40:29.921124+0000","last_clean":"2024-04-16T15:40:29.921124+0000","last_became_active":"2024-04-16T15:40:23.529575+0000","last_became_peered":"2024-04-16T15:40:23.529575+0000","last_unstale":"2024-04-16T15:40:29.921124+0000","last_undegraded":"2024-04-16T15:40:29.921124+0000","last_fullsized":"2024-04-16T15:40:29.921124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920486+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:40:29.920486+0000","last_peered":"2024-04-16T15:40:29.920486+0000","last_clean":"2024-04-16T15:40:29.920486+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:40:29.920486+0000","last_undegraded":"2024-04-16T15:40:29.920486+0000","last_fullsized":"2024-04-16T15:40:29.920486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567180+0000","last_change":"2024-04-16T15:40:28.805311+0000","last_active":"2024-04-16T15:40:29.567180+0000","last_peered":"2024-04-16T15:40:29.567180+0000","last_clean":"2024-04-16T15:40:29.567180+0000","last_became_active":"2024-04-16T15:40:28.804868+0000","last_became_peered":"2024-04-16T15:40:28.804868+0000","last_unstale":"2024-04-16T15:40:29.567180+0000","last_undegraded":"2024-04-16T15:40:29.567180+0000","last_fullsized":"2024-04-16T15:40:29.567180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:32.196316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921580+0000","last_change":"2024-04-16T15:40:24.544114+0000","last_active":"2024-04-16T15:40:29.921580+0000","last_peered":"2024-04-16T15:40:29.921580+0000","last_clean":"2024-04-16T15:40:29.921580+0000","last_became_active":"2024-04-16T15:40:24.543798+0000","last_became_peered":"2024-04-16T15:40:24.543798+0000","last_unstale":"2024-04-16T15:40:29.921580+0000","last_undegraded":"2024-04-16T15:40:29.921580+0000","last_fullsized":"2024-04-16T15:40:29.921580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:38.664081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567401+0000","last_change":"2024-04-16T15:40:23.525608+0000","last_active":"2024-04-16T15:40:29.567401+0000","last_peered":"2024-04-16T15:40:29.567401+0000","last_clean":"2024-04-16T15:40:29.567401+0000","last_became_active":"2024-04-16T15:40:23.525361+0000","last_became_peered":"2024-04-16T15:40:23.525361+0000","last_unstale":"2024-04-16T15:40:29.567401+0000","last_undegraded":"2024-04-16T15:40:29.567401+0000","last_fullsized":"2024-04-16T15:40:29.567401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:17.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566561+0000","last_change":"2024-04-16T15:40:28.806076+0000","last_active":"2024-04-16T15:40:29.566561+0000","last_peered":"2024-04-16T15:40:29.566561+0000","last_clean":"2024-04-16T15:40:29.566561+0000","last_became_active":"2024-04-16T15:40:28.805387+0000","last_became_peered":"2024-04-16T15:40:28.805387+0000","last_unstale":"2024-04-16T15:40:29.566561+0000","last_undegraded":"2024-04-16T15:40:29.566561+0000","last_fullsized":"2024-04-16T15:40:29.566561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:55.765984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920030+0000","last_change":"2024-04-16T15:40:24.546430+0000","last_active":"2024-04-16T15:40:29.920030+0000","last_peered":"2024-04-16T15:40:29.920030+0000","last_clean":"2024-04-16T15:40:29.920030+0000","last_became_active":"2024-04-16T15:40:24.546177+0000","last_became_peered":"2024-04-16T15:40:24.546177+0000","last_unstale":"2024-04-16T15:40:29.920030+0000","last_undegraded":"2024-04-16T15:40:29.920030+0000","last_fullsized":"2024-04-16T15:40:29.920030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:00.952582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769778+0000","last_change":"2024-04-16T15:40:23.537830+0000","last_active":"2024-04-16T15:40:29.769778+0000","last_peered":"2024-04-16T15:40:29.769778+0000","last_clean":"2024-04-16T15:40:29.769778+0000","last_became_active":"2024-04-16T15:40:23.537574+0000","last_became_peered":"2024-04-16T15:40:23.537574+0000","last_unstale":"2024-04-16T15:40:29.769778+0000","last_undegraded":"2024-04-16T15:40:29.769778+0000","last_fullsized":"2024-04-16T15:40:29.769778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918912+0000","last_change":"2024-04-16T15:40:28.571485+0000","last_active":"2024-04-16T15:40:29.918912+0000","last_peered":"2024-04-16T15:40:29.918912+0000","last_clean":"2024-04-16T15:40:29.918912+0000","last_became_active":"2024-04-16T15:40:28.570667+0000","last_became_peered":"2024-04-16T15:40:28.570667+0000","last_unstale":"2024-04-16T15:40:29.918912+0000","last_undegraded":"2024-04-16T15:40:29.918912+0000","last_fullsized":"2024-04-16T15:40:29.918912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:04.979453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920895+0000","last_change":"2024-04-16T15:40:24.548267+0000","last_active":"2024-04-16T15:40:29.920895+0000","last_peered":"2024-04-16T15:40:29.920895+0000","last_clean":"2024-04-16T15:40:29.920895+0000","last_became_active":"2024-04-16T15:40:24.547973+0000","last_became_peered":"2024-04-16T15:40:24.547973+0000","last_unstale":"2024-04-16T15:40:29.920895+0000","last_undegraded":"2024-04-16T15:40:29.920895+0000","last_fullsized":"2024-04-16T15:40:29.920895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:50.186857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566718+0000","last_change":"2024-04-16T15:40:23.523969+0000","last_active":"2024-04-16T15:40:29.566718+0000","last_peered":"2024-04-16T15:40:29.566718+0000","last_clean":"2024-04-16T15:40:29.566718+0000","last_became_active":"2024-04-16T15:40:23.523676+0000","last_became_peered":"2024-04-16T15:40:23.523676+0000","last_unstale":"2024-04-16T15:40:29.566718+0000","last_undegraded":"2024-04-16T15:40:29.566718+0000","last_fullsized":"2024-04-16T15:40:29.566718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919411+0000","last_change":"2024-04-16T15:40:28.807519+0000","last_active":"2024-04-16T15:40:29.919411+0000","last_peered":"2024-04-16T15:40:29.919411+0000","last_clean":"2024-04-16T15:40:29.919411+0000","last_became_active":"2024-04-16T15:40:28.807122+0000","last_became_peered":"2024-04-16T15:40:28.807122+0000","last_unstale":"2024-04-16T15:40:29.919411+0000","last_undegraded":"2024-04-16T15:40:29.919411+0000","last_fullsized":"2024-04-16T15:40:29.919411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:52:28.865900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920489+0000","last_change":"2024-04-16T15:40:24.545147+0000","last_active":"2024-04-16T15:40:29.920489+0000","last_peered":"2024-04-16T15:40:29.920489+0000","last_clean":"2024-04-16T15:40:29.920489+0000","last_became_active":"2024-04-16T15:40:24.544857+0000","last_became_peered":"2024-04-16T15:40:24.544857+0000","last_unstale":"2024-04-16T15:40:29.920489+0000","last_undegraded":"2024-04-16T15:40:29.920489+0000","last_fullsized":"2024-04-16T15:40:29.920489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:31.030716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920619+0000","last_change":"2024-04-16T15:40:23.536928+0000","last_active":"2024-04-16T15:40:29.920619+0000","last_peered":"2024-04-16T15:40:29.920619+0000","last_clean":"2024-04-16T15:40:29.920619+0000","last_became_active":"2024-04-16T15:40:23.535290+0000","last_became_peered":"2024-04-16T15:40:23.535290+0000","last_unstale":"2024-04-16T15:40:29.920619+0000","last_undegraded":"2024-04-16T15:40:29.920619+0000","last_fullsized":"2024-04-16T15:40:29.920619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:26.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919286+0000","last_change":"2024-04-16T15:40:28.561919+0000","last_active":"2024-04-16T15:40:29.919286+0000","last_peered":"2024-04-16T15:40:29.919286+0000","last_clean":"2024-04-16T15:40:29.919286+0000","last_became_active":"2024-04-16T15:40:28.561491+0000","last_became_peered":"2024-04-16T15:40:28.561491+0000","last_unstale":"2024-04-16T15:40:29.919286+0000","last_undegraded":"2024-04-16T15:40:29.919286+0000","last_fullsized":"2024-04-16T15:40:29.919286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:19.925886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919818+0000","last_change":"2024-04-16T15:40:24.544255+0000","last_active":"2024-04-16T15:40:29.919818+0000","last_peered":"2024-04-16T15:40:29.919818+0000","last_clean":"2024-04-16T15:40:29.919818+0000","last_became_active":"2024-04-16T15:40:24.543995+0000","last_became_peered":"2024-04-16T15:40:24.543995+0000","last_unstale":"2024-04-16T15:40:29.919818+0000","last_undegraded":"2024-04-16T15:40:29.919818+0000","last_fullsized":"2024-04-16T15:40:29.919818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:28.169445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566390+0000","last_change":"2024-04-16T15:40:23.530042+0000","last_active":"2024-04-16T15:40:29.566390+0000","last_peered":"2024-04-16T15:40:29.566390+0000","last_clean":"2024-04-16T15:40:29.566390+0000","last_became_active":"2024-04-16T15:40:23.528762+0000","last_became_peered":"2024-04-16T15:40:23.528762+0000","last_unstale":"2024-04-16T15:40:29.566390+0000","last_undegraded":"2024-04-16T15:40:29.566390+0000","last_fullsized":"2024-04-16T15:40:29.566390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919066+0000","last_change":"2024-04-16T15:40:28.568830+0000","last_active":"2024-04-16T15:40:29.919066+0000","last_peered":"2024-04-16T15:40:29.919066+0000","last_clean":"2024-04-16T15:40:29.919066+0000","last_became_active":"2024-04-16T15:40:28.568395+0000","last_became_peered":"2024-04-16T15:40:28.568395+0000","last_unstale":"2024-04-16T15:40:29.919066+0000","last_undegraded":"2024-04-16T15:40:29.919066+0000","last_fullsized":"2024-04-16T15:40:29.919066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:42.690952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566645+0000","last_change":"2024-04-16T15:40:24.543975+0000","last_active":"2024-04-16T15:40:29.566645+0000","last_peered":"2024-04-16T15:40:29.566645+0000","last_clean":"2024-04-16T15:40:29.566645+0000","last_became_active":"2024-04-16T15:40:24.543720+0000","last_became_peered":"2024-04-16T15:40:24.543720+0000","last_unstale":"2024-04-16T15:40:29.566645+0000","last_undegraded":"2024-04-16T15:40:29.566645+0000","last_fullsized":"2024-04-16T15:40:29.566645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566771+0000","last_change":"2024-04-16T15:40:23.536758+0000","last_active":"2024-04-16T15:40:29.566771+0000","last_peered":"2024-04-16T15:40:29.566771+0000","last_clean":"2024-04-16T15:40:29.566771+0000","last_became_active":"2024-04-16T15:40:23.536479+0000","last_became_peered":"2024-04-16T15:40:23.536479+0000","last_unstale":"2024-04-16T15:40:29.566771+0000","last_undegraded":"2024-04-16T15:40:29.566771+0000","last_fullsized":"2024-04-16T15:40:29.566771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567798+0000","last_change":"2024-04-16T15:40:28.807067+0000","last_active":"2024-04-16T15:40:29.567798+0000","last_peered":"2024-04-16T15:40:29.567798+0000","last_clean":"2024-04-16T15:40:29.567798+0000","last_became_active":"2024-04-16T15:40:28.805886+0000","last_became_peered":"2024-04-16T15:40:28.805886+0000","last_unstale":"2024-04-16T15:40:29.567798+0000","last_undegraded":"2024-04-16T15:40:29.567798+0000","last_fullsized":"2024-04-16T15:40:29.567798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:52.503196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920077+0000","last_change":"2024-04-16T15:40:24.546841+0000","last_active":"2024-04-16T15:40:29.920077+0000","last_peered":"2024-04-16T15:40:29.920077+0000","last_clean":"2024-04-16T15:40:29.920077+0000","last_became_active":"2024-04-16T15:40:24.546578+0000","last_became_peered":"2024-04-16T15:40:24.546578+0000","last_unstale":"2024-04-16T15:40:29.920077+0000","last_undegraded":"2024-04-16T15:40:29.920077+0000","last_fullsized":"2024-04-16T15:40:29.920077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:00.952582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920432+0000","last_change":"2024-04-16T15:40:23.527740+0000","last_active":"2024-04-16T15:40:29.920432+0000","last_peered":"2024-04-16T15:40:29.920432+0000","last_clean":"2024-04-16T15:40:29.920432+0000","last_became_active":"2024-04-16T15:40:23.527465+0000","last_became_peered":"2024-04-16T15:40:23.527465+0000","last_unstale":"2024-04-16T15:40:29.920432+0000","last_undegraded":"2024-04-16T15:40:29.920432+0000","last_fullsized":"2024-04-16T15:40:29.920432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567222+0000","last_change":"2024-04-16T15:40:28.804586+0000","last_active":"2024-04-16T15:40:29.567222+0000","last_peered":"2024-04-16T15:40:29.567222+0000","last_clean":"2024-04-16T15:40:29.567222+0000","last_became_active":"2024-04-16T15:40:28.804188+0000","last_became_peered":"2024-04-16T15:40:28.804188+0000","last_unstale":"2024-04-16T15:40:29.567222+0000","last_undegraded":"2024-04-16T15:40:29.567222+0000","last_fullsized":"2024-04-16T15:40:29.567222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:54:44.467991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918678+0000","last_change":"2024-04-16T15:40:24.548554+0000","last_active":"2024-04-16T15:40:29.918678+0000","last_peered":"2024-04-16T15:40:29.918678+0000","last_clean":"2024-04-16T15:40:29.918678+0000","last_became_active":"2024-04-16T15:40:24.548292+0000","last_became_peered":"2024-04-16T15:40:24.548292+0000","last_unstale":"2024-04-16T15:40:29.918678+0000","last_undegraded":"2024-04-16T15:40:29.918678+0000","last_fullsized":"2024-04-16T15:40:29.918678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918961+0000","last_change":"2024-04-16T15:40:23.537872+0000","last_active":"2024-04-16T15:40:29.918961+0000","last_peered":"2024-04-16T15:40:29.918961+0000","last_clean":"2024-04-16T15:40:29.918961+0000","last_became_active":"2024-04-16T15:40:23.537586+0000","last_became_peered":"2024-04-16T15:40:23.537586+0000","last_unstale":"2024-04-16T15:40:29.918961+0000","last_undegraded":"2024-04-16T15:40:29.918961+0000","last_fullsized":"2024-04-16T15:40:29.918961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:35.227712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920389+0000","last_change":"2024-04-16T15:40:28.805253+0000","last_active":"2024-04-16T15:40:29.920389+0000","last_peered":"2024-04-16T15:40:29.920389+0000","last_clean":"2024-04-16T15:40:29.920389+0000","last_became_active":"2024-04-16T15:40:28.804927+0000","last_became_peered":"2024-04-16T15:40:28.804927+0000","last_unstale":"2024-04-16T15:40:29.920389+0000","last_undegraded":"2024-04-16T15:40:29.920389+0000","last_fullsized":"2024-04-16T15:40:29.920389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:55.036666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921526+0000","last_change":"2024-04-16T15:40:24.549880+0000","last_active":"2024-04-16T15:40:29.921526+0000","last_peered":"2024-04-16T15:40:29.921526+0000","last_clean":"2024-04-16T15:40:29.921526+0000","last_became_active":"2024-04-16T15:40:24.549176+0000","last_became_peered":"2024-04-16T15:40:24.549176+0000","last_unstale":"2024-04-16T15:40:29.921526+0000","last_undegraded":"2024-04-16T15:40:29.921526+0000","last_fullsized":"2024-04-16T15:40:29.921526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:32.578262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:33.600447+0000","last_change":"2024-04-16T15:40:23.528022+0000","last_active":"2024-04-16T15:40:33.600447+0000","last_peered":"2024-04-16T15:40:33.600447+0000","last_clean":"2024-04-16T15:40:33.600447+0000","last_became_active":"2024-04-16T15:40:23.527723+0000","last_became_peered":"2024-04-16T15:40:23.527723+0000","last_unstale":"2024-04-16T15:40:33.600447+0000","last_undegraded":"2024-04-16T15:40:33.600447+0000","last_fullsized":"2024-04-16T15:40:33.600447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769345+0000","last_change":"2024-04-16T15:40:28.570589+0000","last_active":"2024-04-16T15:40:29.769345+0000","last_peered":"2024-04-16T15:40:29.769345+0000","last_clean":"2024-04-16T15:40:29.769345+0000","last_became_active":"2024-04-16T15:40:28.570223+0000","last_became_peered":"2024-04-16T15:40:28.570223+0000","last_unstale":"2024-04-16T15:40:29.769345+0000","last_undegraded":"2024-04-16T15:40:29.769345+0000","last_fullsized":"2024-04-16T15:40:29.769345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:35.057588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920264+0000","last_change":"2024-04-16T15:40:24.548751+0000","last_active":"2024-04-16T15:40:29.920264+0000","last_peered":"2024-04-16T15:40:29.920264+0000","last_clean":"2024-04-16T15:40:29.920264+0000","last_became_active":"2024-04-16T15:40:24.548459+0000","last_became_peered":"2024-04-16T15:40:24.548459+0000","last_unstale":"2024-04-16T15:40:29.920264+0000","last_undegraded":"2024-04-16T15:40:29.920264+0000","last_fullsized":"2024-04-16T15:40:29.920264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:22.947852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769687+0000","last_change":"2024-04-16T15:40:23.531085+0000","last_active":"2024-04-16T15:40:29.769687+0000","last_peered":"2024-04-16T15:40:29.769687+0000","last_clean":"2024-04-16T15:40:29.769687+0000","last_became_active":"2024-04-16T15:40:23.530814+0000","last_became_peered":"2024-04-16T15:40:23.530814+0000","last_unstale":"2024-04-16T15:40:29.769687+0000","last_undegraded":"2024-04-16T15:40:29.769687+0000","last_fullsized":"2024-04-16T15:40:29.769687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920980+0000","last_change":"2024-04-16T15:40:28.570865+0000","last_active":"2024-04-16T15:40:29.920980+0000","last_peered":"2024-04-16T15:40:29.920980+0000","last_clean":"2024-04-16T15:40:29.920980+0000","last_became_active":"2024-04-16T15:40:28.570475+0000","last_became_peered":"2024-04-16T15:40:28.570475+0000","last_unstale":"2024-04-16T15:40:29.920980+0000","last_undegraded":"2024-04-16T15:40:29.920980+0000","last_fullsized":"2024-04-16T15:40:29.920980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:32.196316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919181+0000","last_change":"2024-04-16T15:40:24.543283+0000","last_active":"2024-04-16T15:40:29.919181+0000","last_peered":"2024-04-16T15:40:29.919181+0000","last_clean":"2024-04-16T15:40:29.919181+0000","last_became_active":"2024-04-16T15:40:24.542708+0000","last_became_peered":"2024-04-16T15:40:24.542708+0000","last_unstale":"2024-04-16T15:40:29.919181+0000","last_undegraded":"2024-04-16T15:40:29.919181+0000","last_fullsized":"2024-04-16T15:40:29.919181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920195+0000","last_change":"2024-04-16T15:40:23.530402+0000","last_active":"2024-04-16T15:40:29.920195+0000","last_peered":"2024-04-16T15:40:29.920195+0000","last_clean":"2024-04-16T15:40:29.920195+0000","last_became_active":"2024-04-16T15:40:23.530122+0000","last_became_peered":"2024-04-16T15:40:23.530122+0000","last_unstale":"2024-04-16T15:40:29.920195+0000","last_undegraded":"2024-04-16T15:40:29.920195+0000","last_fullsized":"2024-04-16T15:40:29.920195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566298+0000","last_change":"2024-04-16T15:40:28.563586+0000","last_active":"2024-04-16T15:40:29.566298+0000","last_peered":"2024-04-16T15:40:29.566298+0000","last_clean":"2024-04-16T15:40:29.566298+0000","last_became_active":"2024-04-16T15:40:28.563187+0000","last_became_peered":"2024-04-16T15:40:28.563187+0000","last_unstale":"2024-04-16T15:40:29.566298+0000","last_undegraded":"2024-04-16T15:40:29.566298+0000","last_fullsized":"2024-04-16T15:40:29.566298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:28.407124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919923+0000","last_change":"2024-04-16T15:40:24.550624+0000","last_active":"2024-04-16T15:40:29.919923+0000","last_peered":"2024-04-16T15:40:29.919923+0000","last_clean":"2024-04-16T15:40:29.919923+0000","last_became_active":"2024-04-16T15:40:24.550410+0000","last_became_peered":"2024-04-16T15:40:24.550410+0000","last_unstale":"2024-04-16T15:40:29.919923+0000","last_undegraded":"2024-04-16T15:40:29.919923+0000","last_fullsized":"2024-04-16T15:40:29.919923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:32.578262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567285+0000","last_change":"2024-04-16T15:40:23.531043+0000","last_active":"2024-04-16T15:40:29.567285+0000","last_peered":"2024-04-16T15:40:29.567285+0000","last_clean":"2024-04-16T15:40:29.567285+0000","last_became_active":"2024-04-16T15:40:23.530764+0000","last_became_peered":"2024-04-16T15:40:23.530764+0000","last_unstale":"2024-04-16T15:40:29.567285+0000","last_undegraded":"2024-04-16T15:40:29.567285+0000","last_fullsized":"2024-04-16T15:40:29.567285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566507+0000","last_change":"2024-04-16T15:40:28.805696+0000","last_active":"2024-04-16T15:40:29.566507+0000","last_peered":"2024-04-16T15:40:29.566507+0000","last_clean":"2024-04-16T15:40:29.566507+0000","last_became_active":"2024-04-16T15:40:28.805319+0000","last_became_peered":"2024-04-16T15:40:28.805319+0000","last_unstale":"2024-04-16T15:40:29.566507+0000","last_undegraded":"2024-04-16T15:40:29.566507+0000","last_fullsized":"2024-04-16T15:40:29.566507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:55.036666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918914+0000","last_change":"2024-04-16T15:40:28.804646+0000","last_active":"2024-04-16T15:40:29.918914+0000","last_peered":"2024-04-16T15:40:29.918914+0000","last_clean":"2024-04-16T15:40:29.918914+0000","last_became_active":"2024-04-16T15:40:28.804239+0000","last_became_peered":"2024-04-16T15:40:28.804239+0000","last_unstale":"2024-04-16T15:40:29.918914+0000","last_undegraded":"2024-04-16T15:40:29.918914+0000","last_fullsized":"2024-04-16T15:40:29.918914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:36.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769103+0000","last_change":"2024-04-16T15:40:24.533204+0000","last_active":"2024-04-16T15:40:29.769103+0000","last_peered":"2024-04-16T15:40:29.769103+0000","last_clean":"2024-04-16T15:40:29.769103+0000","last_became_active":"2024-04-16T15:40:24.532947+0000","last_became_peered":"2024-04-16T15:40:24.532947+0000","last_unstale":"2024-04-16T15:40:29.769103+0000","last_undegraded":"2024-04-16T15:40:29.769103+0000","last_fullsized":"2024-04-16T15:40:29.769103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919749+0000","last_change":"2024-04-16T15:40:23.537693+0000","last_active":"2024-04-16T15:40:29.919749+0000","last_peered":"2024-04-16T15:40:29.919749+0000","last_clean":"2024-04-16T15:40:29.919749+0000","last_became_active":"2024-04-16T15:40:23.536396+0000","last_became_peered":"2024-04-16T15:40:23.536396+0000","last_unstale":"2024-04-16T15:40:29.919749+0000","last_undegraded":"2024-04-16T15:40:29.919749+0000","last_fullsized":"2024-04-16T15:40:29.919749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567324+0000","last_change":"2024-04-16T15:40:28.567889+0000","last_active":"2024-04-16T15:40:29.567324+0000","last_peered":"2024-04-16T15:40:29.567324+0000","last_clean":"2024-04-16T15:40:29.567324+0000","last_became_active":"2024-04-16T15:40:28.567519+0000","last_became_peered":"2024-04-16T15:40:28.567519+0000","last_unstale":"2024-04-16T15:40:29.567324+0000","last_undegraded":"2024-04-16T15:40:29.567324+0000","last_fullsized":"2024-04-16T15:40:29.567324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566216+0000","last_change":"2024-04-16T15:40:24.550785+0000","last_active":"2024-04-16T15:40:29.566216+0000","last_peered":"2024-04-16T15:40:29.566216+0000","last_clean":"2024-04-16T15:40:29.566216+0000","last_became_active":"2024-04-16T15:40:24.550492+0000","last_became_peered":"2024-04-16T15:40:24.550492+0000","last_unstale":"2024-04-16T15:40:29.566216+0000","last_undegraded":"2024-04-16T15:40:29.566216+0000","last_fullsized":"2024-04-16T15:40:29.566216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567393+0000","last_change":"2024-04-16T15:40:23.527649+0000","last_active":"2024-04-16T15:40:29.567393+0000","last_peered":"2024-04-16T15:40:29.567393+0000","last_clean":"2024-04-16T15:40:29.567393+0000","last_became_active":"2024-04-16T15:40:23.526805+0000","last_became_peered":"2024-04-16T15:40:23.526805+0000","last_unstale":"2024-04-16T15:40:29.567393+0000","last_undegraded":"2024-04-16T15:40:29.567393+0000","last_fullsized":"2024-04-16T15:40:29.567393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:17.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566914+0000","last_change":"2024-04-16T15:40:23.536732+0000","last_active":"2024-04-16T15:40:29.566914+0000","last_peered":"2024-04-16T15:40:29.566914+0000","last_clean":"2024-04-16T15:40:29.566914+0000","last_became_active":"2024-04-16T15:40:23.536431+0000","last_became_peered":"2024-04-16T15:40:23.536431+0000","last_unstale":"2024-04-16T15:40:29.566914+0000","last_undegraded":"2024-04-16T15:40:29.566914+0000","last_fullsized":"2024-04-16T15:40:29.566914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918955+0000","last_change":"2024-04-16T15:40:24.538554+0000","last_active":"2024-04-16T15:40:29.918955+0000","last_peered":"2024-04-16T15:40:29.918955+0000","last_clean":"2024-04-16T15:40:29.918955+0000","last_became_active":"2024-04-16T15:40:24.538271+0000","last_became_peered":"2024-04-16T15:40:24.538271+0000","last_unstale":"2024-04-16T15:40:29.918955+0000","last_undegraded":"2024-04-16T15:40:29.918955+0000","last_fullsized":"2024-04-16T15:40:29.918955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567207+0000","last_change":"2024-04-16T15:40:28.566595+0000","last_active":"2024-04-16T15:40:29.567207+0000","last_peered":"2024-04-16T15:40:29.567207+0000","last_clean":"2024-04-16T15:40:29.567207+0000","last_became_active":"2024-04-16T15:40:28.566177+0000","last_became_peered":"2024-04-16T15:40:28.566177+0000","last_unstale":"2024-04-16T15:40:29.567207+0000","last_undegraded":"2024-04-16T15:40:29.567207+0000","last_fullsized":"2024-04-16T15:40:29.567207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:34.092734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567347+0000","last_change":"2024-04-16T15:40:23.528855+0000","last_active":"2024-04-16T15:40:29.567347+0000","last_peered":"2024-04-16T15:40:29.567347+0000","last_clean":"2024-04-16T15:40:29.567347+0000","last_became_active":"2024-04-16T15:40:23.528616+0000","last_became_peered":"2024-04-16T15:40:23.528616+0000","last_unstale":"2024-04-16T15:40:29.567347+0000","last_undegraded":"2024-04-16T15:40:29.567347+0000","last_fullsized":"2024-04-16T15:40:29.567347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919604+0000","last_change":"2024-04-16T15:40:24.535332+0000","last_active":"2024-04-16T15:40:29.919604+0000","last_peered":"2024-04-16T15:40:29.919604+0000","last_clean":"2024-04-16T15:40:29.919604+0000","last_became_active":"2024-04-16T15:40:24.535031+0000","last_became_peered":"2024-04-16T15:40:24.535031+0000","last_unstale":"2024-04-16T15:40:29.919604+0000","last_undegraded":"2024-04-16T15:40:29.919604+0000","last_fullsized":"2024-04-16T15:40:29.919604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:45.872510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567748+0000","last_change":"2024-04-16T15:40:28.564309+0000","last_active":"2024-04-16T15:40:29.567748+0000","last_peered":"2024-04-16T15:40:29.567748+0000","last_clean":"2024-04-16T15:40:29.567748+0000","last_became_active":"2024-04-16T15:40:28.563916+0000","last_became_peered":"2024-04-16T15:40:28.563916+0000","last_unstale":"2024-04-16T15:40:29.567748+0000","last_undegraded":"2024-04-16T15:40:29.567748+0000","last_fullsized":"2024-04-16T15:40:29.567748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:02:28.301124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566423+0000","last_change":"2024-04-16T15:40:24.543242+0000","last_active":"2024-04-16T15:40:29.566423+0000","last_peered":"2024-04-16T15:40:29.566423+0000","last_clean":"2024-04-16T15:40:29.566423+0000","last_became_active":"2024-04-16T15:40:24.542947+0000","last_became_peered":"2024-04-16T15:40:24.542947+0000","last_unstale":"2024-04-16T15:40:29.566423+0000","last_undegraded":"2024-04-16T15:40:29.566423+0000","last_fullsized":"2024-04-16T15:40:29.566423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566822+0000","last_change":"2024-04-16T15:40:24.535196+0000","last_active":"2024-04-16T15:40:29.566822+0000","last_peered":"2024-04-16T15:40:29.566822+0000","last_clean":"2024-04-16T15:40:29.566822+0000","last_became_active":"2024-04-16T15:40:24.534885+0000","last_became_peered":"2024-04-16T15:40:24.534885+0000","last_unstale":"2024-04-16T15:40:29.566822+0000","last_undegraded":"2024-04-16T15:40:29.566822+0000","last_fullsized":"2024-04-16T15:40:29.566822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:20.979138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920587+0000","last_change":"2024-04-16T15:40:24.548666+0000","last_active":"2024-04-16T15:40:29.920587+0000","last_peered":"2024-04-16T15:40:29.920587+0000","last_clean":"2024-04-16T15:40:29.920587+0000","last_became_active":"2024-04-16T15:40:24.548388+0000","last_became_peered":"2024-04-16T15:40:24.548388+0000","last_unstale":"2024-04-16T15:40:29.920587+0000","last_undegraded":"2024-04-16T15:40:29.920587+0000","last_fullsized":"2024-04-16T15:40:29.920587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:28.169445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918853+0000","last_change":"2024-04-16T15:40:24.542842+0000","last_active":"2024-04-16T15:40:29.918853+0000","last_peered":"2024-04-16T15:40:29.918853+0000","last_clean":"2024-04-16T15:40:29.918853+0000","last_became_active":"2024-04-16T15:40:24.542514+0000","last_became_peered":"2024-04-16T15:40:24.542514+0000","last_unstale":"2024-04-16T15:40:29.918853+0000","last_undegraded":"2024-04-16T15:40:29.918853+0000","last_fullsized":"2024-04-16T15:40:29.918853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567390+0000","last_change":"2024-04-16T15:40:24.543863+0000","last_active":"2024-04-16T15:40:29.567390+0000","last_peered":"2024-04-16T15:40:29.567390+0000","last_clean":"2024-04-16T15:40:29.567390+0000","last_became_active":"2024-04-16T15:40:24.543611+0000","last_became_peered":"2024-04-16T15:40:24.543611+0000","last_unstale":"2024-04-16T15:40:29.567390+0000","last_undegraded":"2024-04-16T15:40:29.567390+0000","last_fullsized":"2024-04-16T15:40:29.567390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769571+0000","last_change":"2024-04-16T15:40:24.533049+0000","last_active":"2024-04-16T15:40:29.769571+0000","last_peered":"2024-04-16T15:40:29.769571+0000","last_clean":"2024-04-16T15:40:29.769571+0000","last_became_active":"2024-04-16T15:40:24.532782+0000","last_became_peered":"2024-04-16T15:40:24.532782+0000","last_unstale":"2024-04-16T15:40:29.769571+0000","last_undegraded":"2024-04-16T15:40:29.769571+0000","last_fullsized":"2024-04-16T15:40:29.769571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567235+0000","last_change":"2024-04-16T15:40:24.546110+0000","last_active":"2024-04-16T15:40:29.567235+0000","last_peered":"2024-04-16T15:40:29.567235+0000","last_clean":"2024-04-16T15:40:29.567235+0000","last_became_active":"2024-04-16T15:40:24.545660+0000","last_became_peered":"2024-04-16T15:40:24.545660+0000","last_unstale":"2024-04-16T15:40:29.567235+0000","last_undegraded":"2024-04-16T15:40:29.567235+0000","last_fullsized":"2024-04-16T15:40:29.567235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567701+0000","last_change":"2024-04-16T15:40:24.543839+0000","last_active":"2024-04-16T15:40:29.567701+0000","last_peered":"2024-04-16T15:40:29.567701+0000","last_clean":"2024-04-16T15:40:29.567701+0000","last_became_active":"2024-04-16T15:40:24.543560+0000","last_became_peered":"2024-04-16T15:40:24.543560+0000","last_unstale":"2024-04-16T15:40:29.567701+0000","last_undegraded":"2024-04-16T15:40:29.567701+0000","last_fullsized":"2024-04-16T15:40:29.567701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919650+0000","last_change":"2024-04-16T15:40:24.533578+0000","last_active":"2024-04-16T15:40:29.919650+0000","last_peered":"2024-04-16T15:40:29.919650+0000","last_clean":"2024-04-16T15:40:29.919650+0000","last_became_active":"2024-04-16T15:40:24.533335+0000","last_became_peered":"2024-04-16T15:40:24.533335+0000","last_unstale":"2024-04-16T15:40:29.919650+0000","last_undegraded":"2024-04-16T15:40:29.919650+0000","last_fullsized":"2024-04-16T15:40:29.919650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567192+0000","last_change":"2024-04-16T15:40:24.545340+0000","last_active":"2024-04-16T15:40:29.567192+0000","last_peered":"2024-04-16T15:40:29.567192+0000","last_clean":"2024-04-16T15:40:29.567192+0000","last_became_active":"2024-04-16T15:40:24.545096+0000","last_became_peered":"2024-04-16T15:40:24.545096+0000","last_unstale":"2024-04-16T15:40:29.567192+0000","last_undegraded":"2024-04-16T15:40:29.567192+0000","last_fullsized":"2024-04-16T15:40:29.567192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920042+0000","last_change":"2024-04-16T15:40:24.551716+0000","last_active":"2024-04-16T15:40:29.920042+0000","last_peered":"2024-04-16T15:40:29.920042+0000","last_clean":"2024-04-16T15:40:29.920042+0000","last_became_active":"2024-04-16T15:40:24.551360+0000","last_became_peered":"2024-04-16T15:40:24.551360+0000","last_unstale":"2024-04-16T15:40:29.920042+0000","last_undegraded":"2024-04-16T15:40:29.920042+0000","last_fullsized":"2024-04-16T15:40:29.920042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:14.076265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918705+0000","last_change":"2024-04-16T15:40:24.549554+0000","last_active":"2024-04-16T15:40:29.918705+0000","last_peered":"2024-04-16T15:40:29.918705+0000","last_clean":"2024-04-16T15:40:29.918705+0000","last_became_active":"2024-04-16T15:40:24.548848+0000","last_became_peered":"2024-04-16T15:40:24.548848+0000","last_unstale":"2024-04-16T15:40:29.918705+0000","last_undegraded":"2024-04-16T15:40:29.918705+0000","last_fullsized":"2024-04-16T15:40:29.918705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567025+0000","last_change":"2024-04-16T15:40:24.539612+0000","last_active":"2024-04-16T15:40:29.567025+0000","last_peered":"2024-04-16T15:40:29.567025+0000","last_clean":"2024-04-16T15:40:29.567025+0000","last_became_active":"2024-04-16T15:40:24.536911+0000","last_became_peered":"2024-04-16T15:40:24.536911+0000","last_unstale":"2024-04-16T15:40:29.567025+0000","last_undegraded":"2024-04-16T15:40:29.567025+0000","last_fullsized":"2024-04-16T15:40:29.567025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921716+0000","last_change":"2024-04-16T15:40:24.550620+0000","last_active":"2024-04-16T15:40:29.921716+0000","last_peered":"2024-04-16T15:40:29.921716+0000","last_clean":"2024-04-16T15:40:29.921716+0000","last_became_active":"2024-04-16T15:40:24.549587+0000","last_became_peered":"2024-04-16T15:40:24.549587+0000","last_unstale":"2024-04-16T15:40:29.921716+0000","last_undegraded":"2024-04-16T15:40:29.921716+0000","last_fullsized":"2024-04-16T15:40:29.921716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:27.826954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566610+0000","last_change":"2024-04-16T15:40:24.539758+0000","last_active":"2024-04-16T15:40:29.566610+0000","last_peered":"2024-04-16T15:40:29.566610+0000","last_clean":"2024-04-16T15:40:29.566610+0000","last_became_active":"2024-04-16T15:40:24.539525+0000","last_became_peered":"2024-04-16T15:40:24.539525+0000","last_unstale":"2024-04-16T15:40:29.566610+0000","last_undegraded":"2024-04-16T15:40:29.566610+0000","last_fullsized":"2024-04-16T15:40:29.566610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:44.286135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921033+0000","last_change":"2024-04-16T15:40:24.548867+0000","last_active":"2024-04-16T15:40:29.921033+0000","last_peered":"2024-04-16T15:40:29.921033+0000","last_clean":"2024-04-16T15:40:29.921033+0000","last_became_active":"2024-04-16T15:40:24.548585+0000","last_became_peered":"2024-04-16T15:40:24.548585+0000","last_unstale":"2024-04-16T15:40:29.921033+0000","last_undegraded":"2024-04-16T15:40:29.921033+0000","last_fullsized":"2024-04-16T15:40:29.921033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:31.030716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919546+0000","last_change":"2024-04-16T15:40:24.545697+0000","last_active":"2024-04-16T15:40:29.919546+0000","last_peered":"2024-04-16T15:40:29.919546+0000","last_clean":"2024-04-16T15:40:29.919546+0000","last_became_active":"2024-04-16T15:40:24.545409+0000","last_became_peered":"2024-04-16T15:40:24.545409+0000","last_unstale":"2024-04-16T15:40:29.919546+0000","last_undegraded":"2024-04-16T15:40:29.919546+0000","last_fullsized":"2024-04-16T15:40:29.919546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769173+0000","last_change":"2024-04-16T15:40:24.544806+0000","last_active":"2024-04-16T15:40:29.769173+0000","last_peered":"2024-04-16T15:40:29.769173+0000","last_clean":"2024-04-16T15:40:29.769173+0000","last_became_active":"2024-04-16T15:40:24.544550+0000","last_became_peered":"2024-04-16T15:40:24.544550+0000","last_unstale":"2024-04-16T15:40:29.769173+0000","last_undegraded":"2024-04-16T15:40:29.769173+0000","last_fullsized":"2024-04-16T15:40:29.769173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918672+0000","last_change":"2024-04-16T15:40:24.542813+0000","last_active":"2024-04-16T15:40:29.918672+0000","last_peered":"2024-04-16T15:40:29.918672+0000","last_clean":"2024-04-16T15:40:29.918672+0000","last_became_active":"2024-04-16T15:40:24.534241+0000","last_became_peered":"2024-04-16T15:40:24.534241+0000","last_unstale":"2024-04-16T15:40:29.918672+0000","last_undegraded":"2024-04-16T15:40:29.918672+0000","last_fullsized":"2024-04-16T15:40:29.918672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:30.065862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567152+0000","last_change":"2024-04-16T15:40:24.546408+0000","last_active":"2024-04-16T15:40:29.567152+0000","last_peered":"2024-04-16T15:40:29.567152+0000","last_clean":"2024-04-16T15:40:29.567152+0000","last_became_active":"2024-04-16T15:40:24.546168+0000","last_became_peered":"2024-04-16T15:40:24.546168+0000","last_unstale":"2024-04-16T15:40:29.567152+0000","last_undegraded":"2024-04-16T15:40:29.567152+0000","last_fullsized":"2024-04-16T15:40:29.567152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920159+0000","last_change":"2024-04-16T15:40:24.533057+0000","last_active":"2024-04-16T15:40:29.920159+0000","last_peered":"2024-04-16T15:40:29.920159+0000","last_clean":"2024-04-16T15:40:29.920159+0000","last_became_active":"2024-04-16T15:40:24.532732+0000","last_became_peered":"2024-04-16T15:40:24.532732+0000","last_unstale":"2024-04-16T15:40:29.920159+0000","last_undegraded":"2024-04-16T15:40:29.920159+0000","last_fullsized":"2024-04-16T15:40:29.920159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:13.766871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567143+0000","last_change":"2024-04-16T15:40:24.540869+0000","last_active":"2024-04-16T15:40:29.567143+0000","last_peered":"2024-04-16T15:40:29.567143+0000","last_clean":"2024-04-16T15:40:29.567143+0000","last_became_active":"2024-04-16T15:40:24.537498+0000","last_became_peered":"2024-04-16T15:40:24.537498+0000","last_unstale":"2024-04-16T15:40:29.567143+0000","last_undegraded":"2024-04-16T15:40:29.567143+0000","last_fullsized":"2024-04-16T15:40:29.567143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567216+0000","last_change":"2024-04-16T15:40:24.536207+0000","last_active":"2024-04-16T15:40:29.567216+0000","last_peered":"2024-04-16T15:40:29.567216+0000","last_clean":"2024-04-16T15:40:29.567216+0000","last_became_active":"2024-04-16T15:40:24.535940+0000","last_became_peered":"2024-04-16T15:40:24.535940+0000","last_unstale":"2024-04-16T15:40:29.567216+0000","last_undegraded":"2024-04-16T15:40:29.567216+0000","last_fullsized":"2024-04-16T15:40:29.567216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920836+0000","last_change":"2024-04-16T15:40:24.544527+0000","last_active":"2024-04-16T15:40:29.920836+0000","last_peered":"2024-04-16T15:40:29.920836+0000","last_clean":"2024-04-16T15:40:29.920836+0000","last_became_active":"2024-04-16T15:40:24.543702+0000","last_became_peered":"2024-04-16T15:40:24.543702+0000","last_unstale":"2024-04-16T15:40:29.920836+0000","last_undegraded":"2024-04-16T15:40:29.920836+0000","last_fullsized":"2024-04-16T15:40:29.920836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:24.380253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566076+0000","last_change":"2024-04-16T15:40:24.545972+0000","last_active":"2024-04-16T15:40:29.566076+0000","last_peered":"2024-04-16T15:40:29.566076+0000","last_clean":"2024-04-16T15:40:29.566076+0000","last_became_active":"2024-04-16T15:40:24.545726+0000","last_became_peered":"2024-04-16T15:40:24.545726+0000","last_unstale":"2024-04-16T15:40:29.566076+0000","last_undegraded":"2024-04-16T15:40:29.566076+0000","last_fullsized":"2024-04-16T15:40:29.566076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919710+0000","last_change":"2024-04-16T15:40:24.550333+0000","last_active":"2024-04-16T15:40:29.919710+0000","last_peered":"2024-04-16T15:40:29.919710+0000","last_clean":"2024-04-16T15:40:29.919710+0000","last_became_active":"2024-04-16T15:40:24.550063+0000","last_became_peered":"2024-04-16T15:40:24.550063+0000","last_unstale":"2024-04-16T15:40:29.919710+0000","last_undegraded":"2024-04-16T15:40:29.919710+0000","last_fullsized":"2024-04-16T15:40:29.919710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920040+0000","last_change":"2024-04-16T15:40:24.545717+0000","last_active":"2024-04-16T15:40:29.920040+0000","last_peered":"2024-04-16T15:40:29.920040+0000","last_clean":"2024-04-16T15:40:29.920040+0000","last_became_active":"2024-04-16T15:40:24.545445+0000","last_became_peered":"2024-04-16T15:40:24.545445+0000","last_unstale":"2024-04-16T15:40:29.920040+0000","last_undegraded":"2024-04-16T15:40:29.920040+0000","last_fullsized":"2024-04-16T15:40:29.920040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919205+0000","last_change":"2024-04-16T15:40:24.549202+0000","last_active":"2024-04-16T15:40:29.919205+0000","last_peered":"2024-04-16T15:40:29.919205+0000","last_clean":"2024-04-16T15:40:29.919205+0000","last_became_active":"2024-04-16T15:40:24.548794+0000","last_became_peered":"2024-04-16T15:40:24.548794+0000","last_unstale":"2024-04-16T15:40:29.919205+0000","last_undegraded":"2024-04-16T15:40:29.919205+0000","last_fullsized":"2024-04-16T15:40:29.919205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:25.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919742+0000","last_change":"2024-04-16T15:40:24.548634+0000","last_active":"2024-04-16T15:40:29.919742+0000","last_peered":"2024-04-16T15:40:29.919742+0000","last_clean":"2024-04-16T15:40:29.919742+0000","last_became_active":"2024-04-16T15:40:24.548342+0000","last_became_peered":"2024-04-16T15:40:24.548342+0000","last_unstale":"2024-04-16T15:40:29.919742+0000","last_undegraded":"2024-04-16T15:40:29.919742+0000","last_fullsized":"2024-04-16T15:40:29.919742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:18.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918832+0000","last_change":"2024-04-16T15:40:24.534160+0000","last_active":"2024-04-16T15:40:29.918832+0000","last_peered":"2024-04-16T15:40:29.918832+0000","last_clean":"2024-04-16T15:40:29.918832+0000","last_became_active":"2024-04-16T15:40:24.533913+0000","last_became_peered":"2024-04-16T15:40:24.533913+0000","last_unstale":"2024-04-16T15:40:29.918832+0000","last_undegraded":"2024-04-16T15:40:29.918832+0000","last_fullsized":"2024-04-16T15:40:29.918832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769869+0000","last_change":"2024-04-16T15:40:24.546471+0000","last_active":"2024-04-16T15:40:29.769869+0000","last_peered":"2024-04-16T15:40:29.769869+0000","last_clean":"2024-04-16T15:40:29.769869+0000","last_became_active":"2024-04-16T15:40:24.545638+0000","last_became_peered":"2024-04-16T15:40:24.545638+0000","last_unstale":"2024-04-16T15:40:29.769869+0000","last_undegraded":"2024-04-16T15:40:29.769869+0000","last_fullsized":"2024-04-16T15:40:29.769869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920784+0000","last_change":"2024-04-16T15:40:24.546172+0000","last_active":"2024-04-16T15:40:29.920784+0000","last_peered":"2024-04-16T15:40:29.920784+0000","last_clean":"2024-04-16T15:40:29.920784+0000","last_became_active":"2024-04-16T15:40:24.545582+0000","last_became_peered":"2024-04-16T15:40:24.545582+0000","last_unstale":"2024-04-16T15:40:29.920784+0000","last_undegraded":"2024-04-16T15:40:29.920784+0000","last_fullsized":"2024-04-16T15:40:29.920784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:21.801823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:40:41.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:40:41.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.888+0000 7fcbfdc7b640 1 Processor -- start 2024-04-16T15:40:41.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.888+0000 7fcbfdc7b640 1 -- start start 2024-04-16T15:40:41.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbfdc7b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcbf806cbb0 0x7fcbf8072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbfdc7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 0x7fcbf80c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbfdc7b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf80c5b00 con 0x7fcbf80c2fa0 2024-04-16T15:40:41.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbfdc7b640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf80c6390 con 0x7fcbf806cbb0 2024-04-16T15:40:41.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbfdc7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf80c6670 con 0x7fcbf80c3f40 2024-04-16T15:40:41.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbf77fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcbf806cbb0 0x7fcbf8072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbf77fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcbf806cbb0 0x7fcbf8072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46744/0 (socket says 172.21.15.71:46744) 2024-04-16T15:40:41.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbf77fe640 1 -- 172.21.15.71:0/1192609635 learned_addr learned my addr 172.21.15.71:0/1192609635 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:41.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbf6ffd640 1 --2- 172.21.15.71:0/1192609635 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 0x7fcbf80c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbf77fe640 1 -- 172.21.15.71:0/1192609635 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 msgr2=0x7fcbf80c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:41.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbf77fe640 1 --2- 172.21.15.71:0/1192609635 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 0x7fcbf80c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbf77fe640 1 -- 172.21.15.71:0/1192609635 >> v1:172.21.15.122:6789/0 conn(0x7fcbf80c2fa0 legacy=0x7fcbf80c0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.889+0000 7fcbf77fe640 1 -- 172.21.15.71:0/1192609635 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf80c5da0 con 0x7fcbf806cbb0 2024-04-16T15:40:41.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.890+0000 7fcbf77fe640 1 --2- 172.21.15.71:0/1192609635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf8072240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c7e33f8136b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.890+0000 7fcbf67fc640 1 -- 172.21.15.71:0/1192609635 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe802c9e0 con 0x7fcbf806cbb0 2024-04-16T15:40:41.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.890+0000 7fcbf67fc640 1 -- 172.21.15.71:0/1192609635 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe802cb80 con 0x7fcbf806cbb0 2024-04-16T15:40:41.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.890+0000 7fcbf67fc640 1 -- 172.21.15.71:0/1192609635 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80343f0 con 0x7fcbf806cbb0 2024-04-16T15:40:41.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.891+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/1192609635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 msgr2=0x7fcbf8072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.891+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/1192609635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf8072240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009e50 tx=0x7fcbe802be40 comp rx=0 tx=0).stop 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.891+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/1192609635 shutdown_connections 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.891+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/1192609635 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 0x7fcbf80c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.891+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/1192609635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf8072240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.891+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/1192609635 >> 172.21.15.71:0/1192609635 conn(0x7fcbf80c45d0 msgr2=0x7fcbf80c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.891+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/1192609635 shutdown_connections 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.891+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/1192609635 wait complete. 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.892+0000 7fcbfdc7b640 1 Processor -- start 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.892+0000 7fcbfdc7b640 1 -- start start 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbfdc7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf81eea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbfdc7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf80c2fa0 0x7fcbf81dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbfdc7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 0x7fcbf81ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbfdc7b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf806cf90 con 0x7fcbf806cbb0 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbfdc7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8073060 con 0x7fcbf80c2fa0 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbfdc7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf806d110 con 0x7fcbf80c3f40 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf81eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf81eea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46752/0 (socket says 172.21.15.71:46752) 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 -- 172.21.15.71:0/2625112154 learned_addr learned my addr 172.21.15.71:0/2625112154 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf7fff640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 0x7fcbf81ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf6ffd640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf80c2fa0 0x7fcbf81dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 -- 172.21.15.71:0/2625112154 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 msgr2=0x7fcbf81ebc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 0x7fcbf81ebc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 -- 172.21.15.71:0/2625112154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf80c2fa0 msgr2=0x7fcbf81dc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf80c2fa0 0x7fcbf81dc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 -- 172.21.15.71:0/2625112154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf81eb490 con 0x7fcbf806cbb0 2024-04-16T15:40:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.893+0000 7fcbf77fe640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf81eea00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8000f80 tx=0x7fcbe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.894+0000 7fcbdffff640 1 -- 172.21.15.71:0/2625112154 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8043020 con 0x7fcbf806cbb0 2024-04-16T15:40:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.894+0000 7fcbdffff640 1 -- 172.21.15.71:0/2625112154 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe803e450 con 0x7fcbf806cbb0 2024-04-16T15:40:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.894+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf80a6130 con 0x7fcbf806cbb0 2024-04-16T15:40:41.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.894+0000 7fcbdffff640 1 -- 172.21.15.71:0/2625112154 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe803e840 con 0x7fcbf806cbb0 2024-04-16T15:40:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.894+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf809b730 con 0x7fcbf806cbb0 2024-04-16T15:40:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.895+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf80c3b20 con 0x7fcbf806cbb0 2024-04-16T15:40:41.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.895+0000 7fcbdffff640 1 -- 172.21.15.71:0/2625112154 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbe8046020 con 0x7fcbf806cbb0 2024-04-16T15:40:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.895+0000 7fcbdffff640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcbe406c920 0x7fcbe406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.896+0000 7fcbdffff640 1 -- 172.21.15.71:0/2625112154 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcbe80ae6b0 con 0x7fcbf806cbb0 2024-04-16T15:40:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.896+0000 7fcbf6ffd640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcbe406c920 0x7fcbe406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:41.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.896+0000 7fcbf6ffd640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcbe406c920 0x7fcbe406ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcbf81ec5c0 tx=0x7fcbec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:41.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:41.900+0000 7fcbdffff640 1 -- 172.21.15.71:0/2625112154 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbf80c3b20 con 0x7fcbf806cbb0 2024-04-16T15:40:42.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.071+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcbf80a5ee0 con 0x7fcbe406c920 2024-04-16T15:40:42.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.079+0000 7fcbdffff640 1 -- 172.21.15.71:0/2625112154 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fcbf80a5ee0 con 0x7fcbe406c920 2024-04-16T15:40:42.082 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:42.084 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:40:42.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcbe406c920 msgr2=0x7fcbe406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcbe406c920 0x7fcbe406ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcbf81ec5c0 tx=0x7fcbec008040 comp rx=0 tx=0).stop 2024-04-16T15:40:42.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 msgr2=0x7fcbf81eea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf81eea00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8000f80 tx=0x7fcbe8004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:42.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 shutdown_connections 2024-04-16T15:40:42.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcbe406c920 0x7fcbe406ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcbf80c3f40 0x7fcbf81ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbf80c2fa0 0x7fcbf81dc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 --2- 172.21.15.71:0/2625112154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcbf806cbb0 0x7fcbf81eea00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 >> 172.21.15.71:0/2625112154 conn(0x7fcbf80c45d0 msgr2=0x7fcbf81da830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 shutdown_connections 2024-04-16T15:40:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.082+0000 7fcbfdc7b640 1 -- 172.21.15.71:0/2625112154 wait complete. 2024-04-16T15:40:42.093 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-16T15:40:41.698963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219396,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755324,"statfs":{"total":773094113280,"available":772869451776,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002523"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919216+0000","last_change":"2024-04-16T15:40:23.532452+0000","last_active":"2024-04-16T15:40:29.919216+0000","last_peered":"2024-04-16T15:40:29.919216+0000","last_clean":"2024-04-16T15:40:29.919216+0000","last_became_active":"2024-04-16T15:40:23.531709+0000","last_became_peered":"2024-04-16T15:40:23.531709+0000","last_unstale":"2024-04-16T15:40:29.919216+0000","last_undegraded":"2024-04-16T15:40:29.919216+0000","last_fullsized":"2024-04-16T15:40:29.919216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:52.586318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567735+0000","last_change":"2024-04-16T15:40:24.542948+0000","last_active":"2024-04-16T15:40:29.567735+0000","last_peered":"2024-04-16T15:40:29.567735+0000","last_clean":"2024-04-16T15:40:29.567735+0000","last_became_active":"2024-04-16T15:40:24.537898+0000","last_became_peered":"2024-04-16T15:40:24.537898+0000","last_unstale":"2024-04-16T15:40:29.567735+0000","last_undegraded":"2024-04-16T15:40:29.567735+0000","last_fullsized":"2024-04-16T15:40:29.567735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:18.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566318+0000","last_change":"2024-04-16T15:40:28.807073+0000","last_active":"2024-04-16T15:40:29.566318+0000","last_peered":"2024-04-16T15:40:29.566318+0000","last_clean":"2024-04-16T15:40:29.566318+0000","last_became_active":"2024-04-16T15:40:28.806446+0000","last_became_peered":"2024-04-16T15:40:28.806446+0000","last_unstale":"2024-04-16T15:40:29.566318+0000","last_undegraded":"2024-04-16T15:40:29.566318+0000","last_fullsized":"2024-04-16T15:40:29.566318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:54.213728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567940+0000","last_change":"2024-04-16T15:40:23.526932+0000","last_active":"2024-04-16T15:40:29.567940+0000","last_peered":"2024-04-16T15:40:29.567940+0000","last_clean":"2024-04-16T15:40:29.567940+0000","last_became_active":"2024-04-16T15:40:23.526663+0000","last_became_peered":"2024-04-16T15:40:23.526663+0000","last_unstale":"2024-04-16T15:40:29.567940+0000","last_undegraded":"2024-04-16T15:40:29.567940+0000","last_fullsized":"2024-04-16T15:40:29.567940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:52.586318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919103+0000","last_change":"2024-04-16T15:40:24.541524+0000","last_active":"2024-04-16T15:40:29.919103+0000","last_peered":"2024-04-16T15:40:29.919103+0000","last_clean":"2024-04-16T15:40:29.919103+0000","last_became_active":"2024-04-16T15:40:24.540243+0000","last_became_peered":"2024-04-16T15:40:24.540243+0000","last_unstale":"2024-04-16T15:40:29.919103+0000","last_undegraded":"2024-04-16T15:40:29.919103+0000","last_fullsized":"2024-04-16T15:40:29.919103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919188+0000","last_change":"2024-04-16T15:40:28.569991+0000","last_active":"2024-04-16T15:40:29.919188+0000","last_peered":"2024-04-16T15:40:29.919188+0000","last_clean":"2024-04-16T15:40:29.919188+0000","last_became_active":"2024-04-16T15:40:28.569596+0000","last_became_peered":"2024-04-16T15:40:28.569596+0000","last_unstale":"2024-04-16T15:40:29.919188+0000","last_undegraded":"2024-04-16T15:40:29.919188+0000","last_fullsized":"2024-04-16T15:40:29.919188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:21.999059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921685+0000","last_change":"2024-04-16T15:40:23.535294+0000","last_active":"2024-04-16T15:40:29.921685+0000","last_peered":"2024-04-16T15:40:29.921685+0000","last_clean":"2024-04-16T15:40:29.921685+0000","last_became_active":"2024-04-16T15:40:23.534958+0000","last_became_peered":"2024-04-16T15:40:23.534958+0000","last_unstale":"2024-04-16T15:40:29.921685+0000","last_undegraded":"2024-04-16T15:40:29.921685+0000","last_fullsized":"2024-04-16T15:40:29.921685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:59.914744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919928+0000","last_change":"2024-04-16T15:40:24.544556+0000","last_active":"2024-04-16T15:40:29.919928+0000","last_peered":"2024-04-16T15:40:29.919928+0000","last_clean":"2024-04-16T15:40:29.919928+0000","last_became_active":"2024-04-16T15:40:24.544072+0000","last_became_peered":"2024-04-16T15:40:24.544072+0000","last_unstale":"2024-04-16T15:40:29.919928+0000","last_undegraded":"2024-04-16T15:40:29.919928+0000","last_fullsized":"2024-04-16T15:40:29.919928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:25.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920842+0000","last_change":"2024-04-16T15:40:28.570724+0000","last_active":"2024-04-16T15:40:29.920842+0000","last_peered":"2024-04-16T15:40:29.920842+0000","last_clean":"2024-04-16T15:40:29.920842+0000","last_became_active":"2024-04-16T15:40:28.570338+0000","last_became_peered":"2024-04-16T15:40:28.570338+0000","last_unstale":"2024-04-16T15:40:29.920842+0000","last_undegraded":"2024-04-16T15:40:29.920842+0000","last_fullsized":"2024-04-16T15:40:29.920842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:55.765984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920279+0000","last_change":"2024-04-16T15:40:23.536319+0000","last_active":"2024-04-16T15:40:29.920279+0000","last_peered":"2024-04-16T15:40:29.920279+0000","last_clean":"2024-04-16T15:40:29.920279+0000","last_became_active":"2024-04-16T15:40:23.536059+0000","last_became_peered":"2024-04-16T15:40:23.536059+0000","last_unstale":"2024-04-16T15:40:29.920279+0000","last_undegraded":"2024-04-16T15:40:29.920279+0000","last_fullsized":"2024-04-16T15:40:29.920279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:19.974713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920444+0000","last_change":"2024-04-16T15:40:24.543661+0000","last_active":"2024-04-16T15:40:29.920444+0000","last_peered":"2024-04-16T15:40:29.920444+0000","last_clean":"2024-04-16T15:40:29.920444+0000","last_became_active":"2024-04-16T15:40:24.543414+0000","last_became_peered":"2024-04-16T15:40:24.543414+0000","last_unstale":"2024-04-16T15:40:29.920444+0000","last_undegraded":"2024-04-16T15:40:29.920444+0000","last_fullsized":"2024-04-16T15:40:29.920444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:38.664081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919473+0000","last_change":"2024-04-16T15:40:28.566254+0000","last_active":"2024-04-16T15:40:29.919473+0000","last_peered":"2024-04-16T15:40:29.919473+0000","last_clean":"2024-04-16T15:40:29.919473+0000","last_became_active":"2024-04-16T15:40:28.565804+0000","last_became_peered":"2024-04-16T15:40:28.565804+0000","last_unstale":"2024-04-16T15:40:29.919473+0000","last_undegraded":"2024-04-16T15:40:29.919473+0000","last_fullsized":"2024-04-16T15:40:29.919473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:40.008247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566632+0000","last_change":"2024-04-16T15:40:24.539072+0000","last_active":"2024-04-16T15:40:29.566632+0000","last_peered":"2024-04-16T15:40:29.566632+0000","last_clean":"2024-04-16T15:40:29.566632+0000","last_became_active":"2024-04-16T15:40:24.537100+0000","last_became_peered":"2024-04-16T15:40:24.537100+0000","last_unstale":"2024-04-16T15:40:29.566632+0000","last_undegraded":"2024-04-16T15:40:29.566632+0000","last_fullsized":"2024-04-16T15:40:29.566632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920246+0000","last_change":"2024-04-16T15:40:23.537232+0000","last_active":"2024-04-16T15:40:29.920246+0000","last_peered":"2024-04-16T15:40:29.920246+0000","last_clean":"2024-04-16T15:40:29.920246+0000","last_became_active":"2024-04-16T15:40:23.536233+0000","last_became_peered":"2024-04-16T15:40:23.536233+0000","last_unstale":"2024-04-16T15:40:29.920246+0000","last_undegraded":"2024-04-16T15:40:29.920246+0000","last_fullsized":"2024-04-16T15:40:29.920246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:43.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921321+0000","last_change":"2024-04-16T15:40:28.570768+0000","last_active":"2024-04-16T15:40:29.921321+0000","last_peered":"2024-04-16T15:40:29.921321+0000","last_clean":"2024-04-16T15:40:29.921321+0000","last_became_active":"2024-04-16T15:40:28.570416+0000","last_became_peered":"2024-04-16T15:40:28.570416+0000","last_unstale":"2024-04-16T15:40:29.921321+0000","last_undegraded":"2024-04-16T15:40:29.921321+0000","last_fullsized":"2024-04-16T15:40:29.921321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567237+0000","last_change":"2024-04-16T15:40:24.548809+0000","last_active":"2024-04-16T15:40:29.567237+0000","last_peered":"2024-04-16T15:40:29.567237+0000","last_clean":"2024-04-16T15:40:29.567237+0000","last_became_active":"2024-04-16T15:40:24.548463+0000","last_became_peered":"2024-04-16T15:40:24.548463+0000","last_unstale":"2024-04-16T15:40:29.567237+0000","last_undegraded":"2024-04-16T15:40:29.567237+0000","last_fullsized":"2024-04-16T15:40:29.567237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769455+0000","last_change":"2024-04-16T15:40:23.531263+0000","last_active":"2024-04-16T15:40:29.769455+0000","last_peered":"2024-04-16T15:40:29.769455+0000","last_clean":"2024-04-16T15:40:29.769455+0000","last_became_active":"2024-04-16T15:40:23.530998+0000","last_became_peered":"2024-04-16T15:40:23.530998+0000","last_unstale":"2024-04-16T15:40:29.769455+0000","last_undegraded":"2024-04-16T15:40:29.769455+0000","last_fullsized":"2024-04-16T15:40:29.769455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919978+0000","last_change":"2024-04-16T15:40:28.569962+0000","last_active":"2024-04-16T15:40:29.919978+0000","last_peered":"2024-04-16T15:40:29.919978+0000","last_clean":"2024-04-16T15:40:29.919978+0000","last_became_active":"2024-04-16T15:40:28.569572+0000","last_became_peered":"2024-04-16T15:40:28.569572+0000","last_unstale":"2024-04-16T15:40:29.919978+0000","last_undegraded":"2024-04-16T15:40:29.919978+0000","last_fullsized":"2024-04-16T15:40:29.919978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:18.103136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566305+0000","last_change":"2024-04-16T15:40:24.544276+0000","last_active":"2024-04-16T15:40:29.566305+0000","last_peered":"2024-04-16T15:40:29.566305+0000","last_clean":"2024-04-16T15:40:29.566305+0000","last_became_active":"2024-04-16T15:40:24.544035+0000","last_became_peered":"2024-04-16T15:40:24.544035+0000","last_unstale":"2024-04-16T15:40:29.566305+0000","last_undegraded":"2024-04-16T15:40:29.566305+0000","last_fullsized":"2024-04-16T15:40:29.566305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769209+0000","last_change":"2024-04-16T15:40:23.530210+0000","last_active":"2024-04-16T15:40:29.769209+0000","last_peered":"2024-04-16T15:40:29.769209+0000","last_clean":"2024-04-16T15:40:29.769209+0000","last_became_active":"2024-04-16T15:40:23.529955+0000","last_became_peered":"2024-04-16T15:40:23.529955+0000","last_unstale":"2024-04-16T15:40:29.769209+0000","last_undegraded":"2024-04-16T15:40:29.769209+0000","last_fullsized":"2024-04-16T15:40:29.769209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:21.943427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769997+0000","last_change":"2024-04-16T15:40:28.805304+0000","last_active":"2024-04-16T15:40:29.769997+0000","last_peered":"2024-04-16T15:40:29.769997+0000","last_clean":"2024-04-16T15:40:29.769997+0000","last_became_active":"2024-04-16T15:40:28.804926+0000","last_became_peered":"2024-04-16T15:40:28.804926+0000","last_unstale":"2024-04-16T15:40:29.769997+0000","last_undegraded":"2024-04-16T15:40:29.769997+0000","last_fullsized":"2024-04-16T15:40:29.769997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:25.828695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919801+0000","last_change":"2024-04-16T15:40:24.537825+0000","last_active":"2024-04-16T15:40:29.919801+0000","last_peered":"2024-04-16T15:40:29.919801+0000","last_clean":"2024-04-16T15:40:29.919801+0000","last_became_active":"2024-04-16T15:40:24.536478+0000","last_became_peered":"2024-04-16T15:40:24.536478+0000","last_unstale":"2024-04-16T15:40:29.919801+0000","last_undegraded":"2024-04-16T15:40:29.919801+0000","last_fullsized":"2024-04-16T15:40:29.919801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:26.549813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919917+0000","last_change":"2024-04-16T15:40:23.535794+0000","last_active":"2024-04-16T15:40:29.919917+0000","last_peered":"2024-04-16T15:40:29.919917+0000","last_clean":"2024-04-16T15:40:29.919917+0000","last_became_active":"2024-04-16T15:40:23.535373+0000","last_became_peered":"2024-04-16T15:40:23.535373+0000","last_unstale":"2024-04-16T15:40:29.919917+0000","last_undegraded":"2024-04-16T15:40:29.919917+0000","last_fullsized":"2024-04-16T15:40:29.919917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920113+0000","last_change":"2024-04-16T15:40:28.561348+0000","last_active":"2024-04-16T15:40:29.920113+0000","last_peered":"2024-04-16T15:40:29.920113+0000","last_clean":"2024-04-16T15:40:29.920113+0000","last_became_active":"2024-04-16T15:40:28.560833+0000","last_became_peered":"2024-04-16T15:40:28.560833+0000","last_unstale":"2024-04-16T15:40:29.920113+0000","last_undegraded":"2024-04-16T15:40:29.920113+0000","last_fullsized":"2024-04-16T15:40:29.920113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:22.352368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919131+0000","last_change":"2024-04-16T15:40:24.534724+0000","last_active":"2024-04-16T15:40:29.919131+0000","last_peered":"2024-04-16T15:40:29.919131+0000","last_clean":"2024-04-16T15:40:29.919131+0000","last_became_active":"2024-04-16T15:40:24.534469+0000","last_became_peered":"2024-04-16T15:40:24.534469+0000","last_unstale":"2024-04-16T15:40:29.919131+0000","last_undegraded":"2024-04-16T15:40:29.919131+0000","last_fullsized":"2024-04-16T15:40:29.919131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:50:03.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567368+0000","last_change":"2024-04-16T15:40:23.531841+0000","last_active":"2024-04-16T15:40:29.567368+0000","last_peered":"2024-04-16T15:40:29.567368+0000","last_clean":"2024-04-16T15:40:29.567368+0000","last_became_active":"2024-04-16T15:40:23.531565+0000","last_became_peered":"2024-04-16T15:40:23.531565+0000","last_unstale":"2024-04-16T15:40:29.567368+0000","last_undegraded":"2024-04-16T15:40:29.567368+0000","last_fullsized":"2024-04-16T15:40:29.567368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919600+0000","last_change":"2024-04-16T15:40:28.807348+0000","last_active":"2024-04-16T15:40:29.919600+0000","last_peered":"2024-04-16T15:40:29.919600+0000","last_clean":"2024-04-16T15:40:29.919600+0000","last_became_active":"2024-04-16T15:40:28.806984+0000","last_became_peered":"2024-04-16T15:40:28.806984+0000","last_unstale":"2024-04-16T15:40:29.919600+0000","last_undegraded":"2024-04-16T15:40:29.919600+0000","last_fullsized":"2024-04-16T15:40:29.919600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:30.543430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567817+0000","last_change":"2024-04-16T15:40:24.544401+0000","last_active":"2024-04-16T15:40:29.567817+0000","last_peered":"2024-04-16T15:40:29.567817+0000","last_clean":"2024-04-16T15:40:29.567817+0000","last_became_active":"2024-04-16T15:40:24.544156+0000","last_became_peered":"2024-04-16T15:40:24.544156+0000","last_unstale":"2024-04-16T15:40:29.567817+0000","last_undegraded":"2024-04-16T15:40:29.567817+0000","last_fullsized":"2024-04-16T15:40:29.567817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769950+0000","last_change":"2024-04-16T15:40:23.532974+0000","last_active":"2024-04-16T15:40:29.769950+0000","last_peered":"2024-04-16T15:40:29.769950+0000","last_clean":"2024-04-16T15:40:29.769950+0000","last_became_active":"2024-04-16T15:40:23.532692+0000","last_became_peered":"2024-04-16T15:40:23.532692+0000","last_unstale":"2024-04-16T15:40:29.769950+0000","last_undegraded":"2024-04-16T15:40:29.769950+0000","last_fullsized":"2024-04-16T15:40:29.769950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:19.974713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566451+0000","last_change":"2024-04-16T15:40:28.806764+0000","last_active":"2024-04-16T15:40:29.566451+0000","last_peered":"2024-04-16T15:40:29.566451+0000","last_clean":"2024-04-16T15:40:29.566451+0000","last_became_active":"2024-04-16T15:40:28.806437+0000","last_became_peered":"2024-04-16T15:40:28.806437+0000","last_unstale":"2024-04-16T15:40:29.566451+0000","last_undegraded":"2024-04-16T15:40:29.566451+0000","last_fullsized":"2024-04-16T15:40:29.566451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:52:28.865900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769595+0000","last_change":"2024-04-16T15:40:24.536656+0000","last_active":"2024-04-16T15:40:29.769595+0000","last_peered":"2024-04-16T15:40:29.769595+0000","last_clean":"2024-04-16T15:40:29.769595+0000","last_became_active":"2024-04-16T15:40:24.536405+0000","last_became_peered":"2024-04-16T15:40:24.536405+0000","last_unstale":"2024-04-16T15:40:29.769595+0000","last_undegraded":"2024-04-16T15:40:29.769595+0000","last_fullsized":"2024-04-16T15:40:29.769595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921784+0000","last_change":"2024-04-16T15:40:23.536026+0000","last_active":"2024-04-16T15:40:29.921784+0000","last_peered":"2024-04-16T15:40:29.921784+0000","last_clean":"2024-04-16T15:40:29.921784+0000","last_became_active":"2024-04-16T15:40:23.535220+0000","last_became_peered":"2024-04-16T15:40:23.535220+0000","last_unstale":"2024-04-16T15:40:29.921784+0000","last_undegraded":"2024-04-16T15:40:29.921784+0000","last_fullsized":"2024-04-16T15:40:29.921784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:37.570307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918806+0000","last_change":"2024-04-16T15:40:28.804863+0000","last_active":"2024-04-16T15:40:29.918806+0000","last_peered":"2024-04-16T15:40:29.918806+0000","last_clean":"2024-04-16T15:40:29.918806+0000","last_became_active":"2024-04-16T15:40:28.804502+0000","last_became_peered":"2024-04-16T15:40:28.804502+0000","last_unstale":"2024-04-16T15:40:29.918806+0000","last_undegraded":"2024-04-16T15:40:29.918806+0000","last_fullsized":"2024-04-16T15:40:29.918806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920557+0000","last_change":"2024-04-16T15:40:24.551868+0000","last_active":"2024-04-16T15:40:29.920557+0000","last_peered":"2024-04-16T15:40:29.920557+0000","last_clean":"2024-04-16T15:40:29.920557+0000","last_became_active":"2024-04-16T15:40:24.550950+0000","last_became_peered":"2024-04-16T15:40:24.550950+0000","last_unstale":"2024-04-16T15:40:29.920557+0000","last_undegraded":"2024-04-16T15:40:29.920557+0000","last_fullsized":"2024-04-16T15:40:29.920557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:27.826954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769685+0000","last_change":"2024-04-16T15:40:23.533782+0000","last_active":"2024-04-16T15:40:29.769685+0000","last_peered":"2024-04-16T15:40:29.769685+0000","last_clean":"2024-04-16T15:40:29.769685+0000","last_became_active":"2024-04-16T15:40:23.533547+0000","last_became_peered":"2024-04-16T15:40:23.533547+0000","last_unstale":"2024-04-16T15:40:29.769685+0000","last_undegraded":"2024-04-16T15:40:29.769685+0000","last_fullsized":"2024-04-16T15:40:29.769685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769337+0000","last_change":"2024-04-16T15:40:28.570047+0000","last_active":"2024-04-16T15:40:29.769337+0000","last_peered":"2024-04-16T15:40:29.769337+0000","last_clean":"2024-04-16T15:40:29.769337+0000","last_became_active":"2024-04-16T15:40:28.569688+0000","last_became_peered":"2024-04-16T15:40:28.569688+0000","last_unstale":"2024-04-16T15:40:29.769337+0000","last_undegraded":"2024-04-16T15:40:29.769337+0000","last_fullsized":"2024-04-16T15:40:29.769337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:42.690952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921046+0000","last_change":"2024-04-16T15:40:24.549077+0000","last_active":"2024-04-16T15:40:29.921046+0000","last_peered":"2024-04-16T15:40:29.921046+0000","last_clean":"2024-04-16T15:40:29.921046+0000","last_became_active":"2024-04-16T15:40:24.548802+0000","last_became_peered":"2024-04-16T15:40:24.548802+0000","last_unstale":"2024-04-16T15:40:29.921046+0000","last_undegraded":"2024-04-16T15:40:29.921046+0000","last_fullsized":"2024-04-16T15:40:29.921046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:50.142130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920546+0000","last_change":"2024-04-16T15:40:23.528707+0000","last_active":"2024-04-16T15:40:29.920546+0000","last_peered":"2024-04-16T15:40:29.920546+0000","last_clean":"2024-04-16T15:40:29.920546+0000","last_became_active":"2024-04-16T15:40:23.528426+0000","last_became_peered":"2024-04-16T15:40:23.528426+0000","last_unstale":"2024-04-16T15:40:29.920546+0000","last_undegraded":"2024-04-16T15:40:29.920546+0000","last_fullsized":"2024-04-16T15:40:29.920546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769721+0000","last_change":"2024-04-16T15:40:28.561110+0000","last_active":"2024-04-16T15:40:29.769721+0000","last_peered":"2024-04-16T15:40:29.769721+0000","last_clean":"2024-04-16T15:40:29.769721+0000","last_became_active":"2024-04-16T15:40:28.560719+0000","last_became_peered":"2024-04-16T15:40:28.560719+0000","last_unstale":"2024-04-16T15:40:29.769721+0000","last_undegraded":"2024-04-16T15:40:29.769721+0000","last_fullsized":"2024-04-16T15:40:29.769721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:31.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769261+0000","last_change":"2024-04-16T15:40:24.536757+0000","last_active":"2024-04-16T15:40:29.769261+0000","last_peered":"2024-04-16T15:40:29.769261+0000","last_clean":"2024-04-16T15:40:29.769261+0000","last_became_active":"2024-04-16T15:40:24.536520+0000","last_became_peered":"2024-04-16T15:40:24.536520+0000","last_unstale":"2024-04-16T15:40:29.769261+0000","last_undegraded":"2024-04-16T15:40:29.769261+0000","last_fullsized":"2024-04-16T15:40:29.769261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566917+0000","last_change":"2024-04-16T15:40:23.520551+0000","last_active":"2024-04-16T15:40:29.566917+0000","last_peered":"2024-04-16T15:40:29.566917+0000","last_clean":"2024-04-16T15:40:29.566917+0000","last_became_active":"2024-04-16T15:40:23.520300+0000","last_became_peered":"2024-04-16T15:40:23.520300+0000","last_unstale":"2024-04-16T15:40:29.566917+0000","last_undegraded":"2024-04-16T15:40:29.566917+0000","last_fullsized":"2024-04-16T15:40:29.566917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.770097+0000","last_change":"2024-04-16T15:40:28.570553+0000","last_active":"2024-04-16T15:40:29.770097+0000","last_peered":"2024-04-16T15:40:29.770097+0000","last_clean":"2024-04-16T15:40:29.770097+0000","last_became_active":"2024-04-16T15:40:28.569635+0000","last_became_peered":"2024-04-16T15:40:28.569635+0000","last_unstale":"2024-04-16T15:40:29.770097+0000","last_undegraded":"2024-04-16T15:40:29.770097+0000","last_fullsized":"2024-04-16T15:40:29.770097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:28.407124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918519+0000","last_change":"2024-04-16T15:40:24.537795+0000","last_active":"2024-04-16T15:40:29.918519+0000","last_peered":"2024-04-16T15:40:29.918519+0000","last_clean":"2024-04-16T15:40:29.918519+0000","last_became_active":"2024-04-16T15:40:24.537528+0000","last_became_peered":"2024-04-16T15:40:24.537528+0000","last_unstale":"2024-04-16T15:40:29.918519+0000","last_undegraded":"2024-04-16T15:40:29.918519+0000","last_fullsized":"2024-04-16T15:40:29.918519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:38.574731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920406+0000","last_change":"2024-04-16T15:40:23.533872+0000","last_active":"2024-04-16T15:40:29.920406+0000","last_peered":"2024-04-16T15:40:29.920406+0000","last_clean":"2024-04-16T15:40:29.920406+0000","last_became_active":"2024-04-16T15:40:23.533491+0000","last_became_peered":"2024-04-16T15:40:23.533491+0000","last_unstale":"2024-04-16T15:40:29.920406+0000","last_undegraded":"2024-04-16T15:40:29.920406+0000","last_fullsized":"2024-04-16T15:40:29.920406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:21.943427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919078+0000","last_change":"2024-04-16T15:40:28.807602+0000","last_active":"2024-04-16T15:40:29.919078+0000","last_peered":"2024-04-16T15:40:29.919078+0000","last_clean":"2024-04-16T15:40:29.919078+0000","last_became_active":"2024-04-16T15:40:28.807193+0000","last_became_peered":"2024-04-16T15:40:28.807193+0000","last_unstale":"2024-04-16T15:40:29.919078+0000","last_undegraded":"2024-04-16T15:40:29.919078+0000","last_fullsized":"2024-04-16T15:40:29.919078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:35.057588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567071+0000","last_change":"2024-04-16T15:40:24.546548+0000","last_active":"2024-04-16T15:40:29.567071+0000","last_peered":"2024-04-16T15:40:29.567071+0000","last_clean":"2024-04-16T15:40:29.567071+0000","last_became_active":"2024-04-16T15:40:24.546317+0000","last_became_peered":"2024-04-16T15:40:24.546317+0000","last_unstale":"2024-04-16T15:40:29.567071+0000","last_undegraded":"2024-04-16T15:40:29.567071+0000","last_fullsized":"2024-04-16T15:40:29.567071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:14.076265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921124+0000","last_change":"2024-04-16T15:40:23.529866+0000","last_active":"2024-04-16T15:40:29.921124+0000","last_peered":"2024-04-16T15:40:29.921124+0000","last_clean":"2024-04-16T15:40:29.921124+0000","last_became_active":"2024-04-16T15:40:23.529575+0000","last_became_peered":"2024-04-16T15:40:23.529575+0000","last_unstale":"2024-04-16T15:40:29.921124+0000","last_undegraded":"2024-04-16T15:40:29.921124+0000","last_fullsized":"2024-04-16T15:40:29.921124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920486+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:40:29.920486+0000","last_peered":"2024-04-16T15:40:29.920486+0000","last_clean":"2024-04-16T15:40:29.920486+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:40:29.920486+0000","last_undegraded":"2024-04-16T15:40:29.920486+0000","last_fullsized":"2024-04-16T15:40:29.920486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567180+0000","last_change":"2024-04-16T15:40:28.805311+0000","last_active":"2024-04-16T15:40:29.567180+0000","last_peered":"2024-04-16T15:40:29.567180+0000","last_clean":"2024-04-16T15:40:29.567180+0000","last_became_active":"2024-04-16T15:40:28.804868+0000","last_became_peered":"2024-04-16T15:40:28.804868+0000","last_unstale":"2024-04-16T15:40:29.567180+0000","last_undegraded":"2024-04-16T15:40:29.567180+0000","last_fullsized":"2024-04-16T15:40:29.567180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:32.196316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921580+0000","last_change":"2024-04-16T15:40:24.544114+0000","last_active":"2024-04-16T15:40:29.921580+0000","last_peered":"2024-04-16T15:40:29.921580+0000","last_clean":"2024-04-16T15:40:29.921580+0000","last_became_active":"2024-04-16T15:40:24.543798+0000","last_became_peered":"2024-04-16T15:40:24.543798+0000","last_unstale":"2024-04-16T15:40:29.921580+0000","last_undegraded":"2024-04-16T15:40:29.921580+0000","last_fullsized":"2024-04-16T15:40:29.921580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:38.664081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567401+0000","last_change":"2024-04-16T15:40:23.525608+0000","last_active":"2024-04-16T15:40:29.567401+0000","last_peered":"2024-04-16T15:40:29.567401+0000","last_clean":"2024-04-16T15:40:29.567401+0000","last_became_active":"2024-04-16T15:40:23.525361+0000","last_became_peered":"2024-04-16T15:40:23.525361+0000","last_unstale":"2024-04-16T15:40:29.567401+0000","last_undegraded":"2024-04-16T15:40:29.567401+0000","last_fullsized":"2024-04-16T15:40:29.567401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:17.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566561+0000","last_change":"2024-04-16T15:40:28.806076+0000","last_active":"2024-04-16T15:40:29.566561+0000","last_peered":"2024-04-16T15:40:29.566561+0000","last_clean":"2024-04-16T15:40:29.566561+0000","last_became_active":"2024-04-16T15:40:28.805387+0000","last_became_peered":"2024-04-16T15:40:28.805387+0000","last_unstale":"2024-04-16T15:40:29.566561+0000","last_undegraded":"2024-04-16T15:40:29.566561+0000","last_fullsized":"2024-04-16T15:40:29.566561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:55.765984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920030+0000","last_change":"2024-04-16T15:40:24.546430+0000","last_active":"2024-04-16T15:40:29.920030+0000","last_peered":"2024-04-16T15:40:29.920030+0000","last_clean":"2024-04-16T15:40:29.920030+0000","last_became_active":"2024-04-16T15:40:24.546177+0000","last_became_peered":"2024-04-16T15:40:24.546177+0000","last_unstale":"2024-04-16T15:40:29.920030+0000","last_undegraded":"2024-04-16T15:40:29.920030+0000","last_fullsized":"2024-04-16T15:40:29.920030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:00.952582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769778+0000","last_change":"2024-04-16T15:40:23.537830+0000","last_active":"2024-04-16T15:40:29.769778+0000","last_peered":"2024-04-16T15:40:29.769778+0000","last_clean":"2024-04-16T15:40:29.769778+0000","last_became_active":"2024-04-16T15:40:23.537574+0000","last_became_peered":"2024-04-16T15:40:23.537574+0000","last_unstale":"2024-04-16T15:40:29.769778+0000","last_undegraded":"2024-04-16T15:40:29.769778+0000","last_fullsized":"2024-04-16T15:40:29.769778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918912+0000","last_change":"2024-04-16T15:40:28.571485+0000","last_active":"2024-04-16T15:40:29.918912+0000","last_peered":"2024-04-16T15:40:29.918912+0000","last_clean":"2024-04-16T15:40:29.918912+0000","last_became_active":"2024-04-16T15:40:28.570667+0000","last_became_peered":"2024-04-16T15:40:28.570667+0000","last_unstale":"2024-04-16T15:40:29.918912+0000","last_undegraded":"2024-04-16T15:40:29.918912+0000","last_fullsized":"2024-04-16T15:40:29.918912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:04.979453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920895+0000","last_change":"2024-04-16T15:40:24.548267+0000","last_active":"2024-04-16T15:40:29.920895+0000","last_peered":"2024-04-16T15:40:29.920895+0000","last_clean":"2024-04-16T15:40:29.920895+0000","last_became_active":"2024-04-16T15:40:24.547973+0000","last_became_peered":"2024-04-16T15:40:24.547973+0000","last_unstale":"2024-04-16T15:40:29.920895+0000","last_undegraded":"2024-04-16T15:40:29.920895+0000","last_fullsized":"2024-04-16T15:40:29.920895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:50.186857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566718+0000","last_change":"2024-04-16T15:40:23.523969+0000","last_active":"2024-04-16T15:40:29.566718+0000","last_peered":"2024-04-16T15:40:29.566718+0000","last_clean":"2024-04-16T15:40:29.566718+0000","last_became_active":"2024-04-16T15:40:23.523676+0000","last_became_peered":"2024-04-16T15:40:23.523676+0000","last_unstale":"2024-04-16T15:40:29.566718+0000","last_undegraded":"2024-04-16T15:40:29.566718+0000","last_fullsized":"2024-04-16T15:40:29.566718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919411+0000","last_change":"2024-04-16T15:40:28.807519+0000","last_active":"2024-04-16T15:40:29.919411+0000","last_peered":"2024-04-16T15:40:29.919411+0000","last_clean":"2024-04-16T15:40:29.919411+0000","last_became_active":"2024-04-16T15:40:28.807122+0000","last_became_peered":"2024-04-16T15:40:28.807122+0000","last_unstale":"2024-04-16T15:40:29.919411+0000","last_undegraded":"2024-04-16T15:40:29.919411+0000","last_fullsized":"2024-04-16T15:40:29.919411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:52:28.865900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920489+0000","last_change":"2024-04-16T15:40:24.545147+0000","last_active":"2024-04-16T15:40:29.920489+0000","last_peered":"2024-04-16T15:40:29.920489+0000","last_clean":"2024-04-16T15:40:29.920489+0000","last_became_active":"2024-04-16T15:40:24.544857+0000","last_became_peered":"2024-04-16T15:40:24.544857+0000","last_unstale":"2024-04-16T15:40:29.920489+0000","last_undegraded":"2024-04-16T15:40:29.920489+0000","last_fullsized":"2024-04-16T15:40:29.920489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:31.030716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920619+0000","last_change":"2024-04-16T15:40:23.536928+0000","last_active":"2024-04-16T15:40:29.920619+0000","last_peered":"2024-04-16T15:40:29.920619+0000","last_clean":"2024-04-16T15:40:29.920619+0000","last_became_active":"2024-04-16T15:40:23.535290+0000","last_became_peered":"2024-04-16T15:40:23.535290+0000","last_unstale":"2024-04-16T15:40:29.920619+0000","last_undegraded":"2024-04-16T15:40:29.920619+0000","last_fullsized":"2024-04-16T15:40:29.920619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:26.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919286+0000","last_change":"2024-04-16T15:40:28.561919+0000","last_active":"2024-04-16T15:40:29.919286+0000","last_peered":"2024-04-16T15:40:29.919286+0000","last_clean":"2024-04-16T15:40:29.919286+0000","last_became_active":"2024-04-16T15:40:28.561491+0000","last_became_peered":"2024-04-16T15:40:28.561491+0000","last_unstale":"2024-04-16T15:40:29.919286+0000","last_undegraded":"2024-04-16T15:40:29.919286+0000","last_fullsized":"2024-04-16T15:40:29.919286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:19.925886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919818+0000","last_change":"2024-04-16T15:40:24.544255+0000","last_active":"2024-04-16T15:40:29.919818+0000","last_peered":"2024-04-16T15:40:29.919818+0000","last_clean":"2024-04-16T15:40:29.919818+0000","last_became_active":"2024-04-16T15:40:24.543995+0000","last_became_peered":"2024-04-16T15:40:24.543995+0000","last_unstale":"2024-04-16T15:40:29.919818+0000","last_undegraded":"2024-04-16T15:40:29.919818+0000","last_fullsized":"2024-04-16T15:40:29.919818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:28.169445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566390+0000","last_change":"2024-04-16T15:40:23.530042+0000","last_active":"2024-04-16T15:40:29.566390+0000","last_peered":"2024-04-16T15:40:29.566390+0000","last_clean":"2024-04-16T15:40:29.566390+0000","last_became_active":"2024-04-16T15:40:23.528762+0000","last_became_peered":"2024-04-16T15:40:23.528762+0000","last_unstale":"2024-04-16T15:40:29.566390+0000","last_undegraded":"2024-04-16T15:40:29.566390+0000","last_fullsized":"2024-04-16T15:40:29.566390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919066+0000","last_change":"2024-04-16T15:40:28.568830+0000","last_active":"2024-04-16T15:40:29.919066+0000","last_peered":"2024-04-16T15:40:29.919066+0000","last_clean":"2024-04-16T15:40:29.919066+0000","last_became_active":"2024-04-16T15:40:28.568395+0000","last_became_peered":"2024-04-16T15:40:28.568395+0000","last_unstale":"2024-04-16T15:40:29.919066+0000","last_undegraded":"2024-04-16T15:40:29.919066+0000","last_fullsized":"2024-04-16T15:40:29.919066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:42.690952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566645+0000","last_change":"2024-04-16T15:40:24.543975+0000","last_active":"2024-04-16T15:40:29.566645+0000","last_peered":"2024-04-16T15:40:29.566645+0000","last_clean":"2024-04-16T15:40:29.566645+0000","last_became_active":"2024-04-16T15:40:24.543720+0000","last_became_peered":"2024-04-16T15:40:24.543720+0000","last_unstale":"2024-04-16T15:40:29.566645+0000","last_undegraded":"2024-04-16T15:40:29.566645+0000","last_fullsized":"2024-04-16T15:40:29.566645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566771+0000","last_change":"2024-04-16T15:40:23.536758+0000","last_active":"2024-04-16T15:40:29.566771+0000","last_peered":"2024-04-16T15:40:29.566771+0000","last_clean":"2024-04-16T15:40:29.566771+0000","last_became_active":"2024-04-16T15:40:23.536479+0000","last_became_peered":"2024-04-16T15:40:23.536479+0000","last_unstale":"2024-04-16T15:40:29.566771+0000","last_undegraded":"2024-04-16T15:40:29.566771+0000","last_fullsized":"2024-04-16T15:40:29.566771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567798+0000","last_change":"2024-04-16T15:40:28.807067+0000","last_active":"2024-04-16T15:40:29.567798+0000","last_peered":"2024-04-16T15:40:29.567798+0000","last_clean":"2024-04-16T15:40:29.567798+0000","last_became_active":"2024-04-16T15:40:28.805886+0000","last_became_peered":"2024-04-16T15:40:28.805886+0000","last_unstale":"2024-04-16T15:40:29.567798+0000","last_undegraded":"2024-04-16T15:40:29.567798+0000","last_fullsized":"2024-04-16T15:40:29.567798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:52.503196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920077+0000","last_change":"2024-04-16T15:40:24.546841+0000","last_active":"2024-04-16T15:40:29.920077+0000","last_peered":"2024-04-16T15:40:29.920077+0000","last_clean":"2024-04-16T15:40:29.920077+0000","last_became_active":"2024-04-16T15:40:24.546578+0000","last_became_peered":"2024-04-16T15:40:24.546578+0000","last_unstale":"2024-04-16T15:40:29.920077+0000","last_undegraded":"2024-04-16T15:40:29.920077+0000","last_fullsized":"2024-04-16T15:40:29.920077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:00.952582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920432+0000","last_change":"2024-04-16T15:40:23.527740+0000","last_active":"2024-04-16T15:40:29.920432+0000","last_peered":"2024-04-16T15:40:29.920432+0000","last_clean":"2024-04-16T15:40:29.920432+0000","last_became_active":"2024-04-16T15:40:23.527465+0000","last_became_peered":"2024-04-16T15:40:23.527465+0000","last_unstale":"2024-04-16T15:40:29.920432+0000","last_undegraded":"2024-04-16T15:40:29.920432+0000","last_fullsized":"2024-04-16T15:40:29.920432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567222+0000","last_change":"2024-04-16T15:40:28.804586+0000","last_active":"2024-04-16T15:40:29.567222+0000","last_peered":"2024-04-16T15:40:29.567222+0000","last_clean":"2024-04-16T15:40:29.567222+0000","last_became_active":"2024-04-16T15:40:28.804188+0000","last_became_peered":"2024-04-16T15:40:28.804188+0000","last_unstale":"2024-04-16T15:40:29.567222+0000","last_undegraded":"2024-04-16T15:40:29.567222+0000","last_fullsized":"2024-04-16T15:40:29.567222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:54:44.467991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918678+0000","last_change":"2024-04-16T15:40:24.548554+0000","last_active":"2024-04-16T15:40:29.918678+0000","last_peered":"2024-04-16T15:40:29.918678+0000","last_clean":"2024-04-16T15:40:29.918678+0000","last_became_active":"2024-04-16T15:40:24.548292+0000","last_became_peered":"2024-04-16T15:40:24.548292+0000","last_unstale":"2024-04-16T15:40:29.918678+0000","last_undegraded":"2024-04-16T15:40:29.918678+0000","last_fullsized":"2024-04-16T15:40:29.918678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918961+0000","last_change":"2024-04-16T15:40:23.537872+0000","last_active":"2024-04-16T15:40:29.918961+0000","last_peered":"2024-04-16T15:40:29.918961+0000","last_clean":"2024-04-16T15:40:29.918961+0000","last_became_active":"2024-04-16T15:40:23.537586+0000","last_became_peered":"2024-04-16T15:40:23.537586+0000","last_unstale":"2024-04-16T15:40:29.918961+0000","last_undegraded":"2024-04-16T15:40:29.918961+0000","last_fullsized":"2024-04-16T15:40:29.918961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:35.227712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920389+0000","last_change":"2024-04-16T15:40:28.805253+0000","last_active":"2024-04-16T15:40:29.920389+0000","last_peered":"2024-04-16T15:40:29.920389+0000","last_clean":"2024-04-16T15:40:29.920389+0000","last_became_active":"2024-04-16T15:40:28.804927+0000","last_became_peered":"2024-04-16T15:40:28.804927+0000","last_unstale":"2024-04-16T15:40:29.920389+0000","last_undegraded":"2024-04-16T15:40:29.920389+0000","last_fullsized":"2024-04-16T15:40:29.920389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:55.036666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921526+0000","last_change":"2024-04-16T15:40:24.549880+0000","last_active":"2024-04-16T15:40:29.921526+0000","last_peered":"2024-04-16T15:40:29.921526+0000","last_clean":"2024-04-16T15:40:29.921526+0000","last_became_active":"2024-04-16T15:40:24.549176+0000","last_became_peered":"2024-04-16T15:40:24.549176+0000","last_unstale":"2024-04-16T15:40:29.921526+0000","last_undegraded":"2024-04-16T15:40:29.921526+0000","last_fullsized":"2024-04-16T15:40:29.921526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:32.578262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:33.600447+0000","last_change":"2024-04-16T15:40:23.528022+0000","last_active":"2024-04-16T15:40:33.600447+0000","last_peered":"2024-04-16T15:40:33.600447+0000","last_clean":"2024-04-16T15:40:33.600447+0000","last_became_active":"2024-04-16T15:40:23.527723+0000","last_became_peered":"2024-04-16T15:40:23.527723+0000","last_unstale":"2024-04-16T15:40:33.600447+0000","last_undegraded":"2024-04-16T15:40:33.600447+0000","last_fullsized":"2024-04-16T15:40:33.600447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769345+0000","last_change":"2024-04-16T15:40:28.570589+0000","last_active":"2024-04-16T15:40:29.769345+0000","last_peered":"2024-04-16T15:40:29.769345+0000","last_clean":"2024-04-16T15:40:29.769345+0000","last_became_active":"2024-04-16T15:40:28.570223+0000","last_became_peered":"2024-04-16T15:40:28.570223+0000","last_unstale":"2024-04-16T15:40:29.769345+0000","last_undegraded":"2024-04-16T15:40:29.769345+0000","last_fullsized":"2024-04-16T15:40:29.769345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:35.057588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920264+0000","last_change":"2024-04-16T15:40:24.548751+0000","last_active":"2024-04-16T15:40:29.920264+0000","last_peered":"2024-04-16T15:40:29.920264+0000","last_clean":"2024-04-16T15:40:29.920264+0000","last_became_active":"2024-04-16T15:40:24.548459+0000","last_became_peered":"2024-04-16T15:40:24.548459+0000","last_unstale":"2024-04-16T15:40:29.920264+0000","last_undegraded":"2024-04-16T15:40:29.920264+0000","last_fullsized":"2024-04-16T15:40:29.920264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:22.947852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769687+0000","last_change":"2024-04-16T15:40:23.531085+0000","last_active":"2024-04-16T15:40:29.769687+0000","last_peered":"2024-04-16T15:40:29.769687+0000","last_clean":"2024-04-16T15:40:29.769687+0000","last_became_active":"2024-04-16T15:40:23.530814+0000","last_became_peered":"2024-04-16T15:40:23.530814+0000","last_unstale":"2024-04-16T15:40:29.769687+0000","last_undegraded":"2024-04-16T15:40:29.769687+0000","last_fullsized":"2024-04-16T15:40:29.769687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920980+0000","last_change":"2024-04-16T15:40:28.570865+0000","last_active":"2024-04-16T15:40:29.920980+0000","last_peered":"2024-04-16T15:40:29.920980+0000","last_clean":"2024-04-16T15:40:29.920980+0000","last_became_active":"2024-04-16T15:40:28.570475+0000","last_became_peered":"2024-04-16T15:40:28.570475+0000","last_unstale":"2024-04-16T15:40:29.920980+0000","last_undegraded":"2024-04-16T15:40:29.920980+0000","last_fullsized":"2024-04-16T15:40:29.920980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:32.196316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919181+0000","last_change":"2024-04-16T15:40:24.543283+0000","last_active":"2024-04-16T15:40:29.919181+0000","last_peered":"2024-04-16T15:40:29.919181+0000","last_clean":"2024-04-16T15:40:29.919181+0000","last_became_active":"2024-04-16T15:40:24.542708+0000","last_became_peered":"2024-04-16T15:40:24.542708+0000","last_unstale":"2024-04-16T15:40:29.919181+0000","last_undegraded":"2024-04-16T15:40:29.919181+0000","last_fullsized":"2024-04-16T15:40:29.919181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920195+0000","last_change":"2024-04-16T15:40:23.530402+0000","last_active":"2024-04-16T15:40:29.920195+0000","last_peered":"2024-04-16T15:40:29.920195+0000","last_clean":"2024-04-16T15:40:29.920195+0000","last_became_active":"2024-04-16T15:40:23.530122+0000","last_became_peered":"2024-04-16T15:40:23.530122+0000","last_unstale":"2024-04-16T15:40:29.920195+0000","last_undegraded":"2024-04-16T15:40:29.920195+0000","last_fullsized":"2024-04-16T15:40:29.920195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566298+0000","last_change":"2024-04-16T15:40:28.563586+0000","last_active":"2024-04-16T15:40:29.566298+0000","last_peered":"2024-04-16T15:40:29.566298+0000","last_clean":"2024-04-16T15:40:29.566298+0000","last_became_active":"2024-04-16T15:40:28.563187+0000","last_became_peered":"2024-04-16T15:40:28.563187+0000","last_unstale":"2024-04-16T15:40:29.566298+0000","last_undegraded":"2024-04-16T15:40:29.566298+0000","last_fullsized":"2024-04-16T15:40:29.566298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:28.407124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919923+0000","last_change":"2024-04-16T15:40:24.550624+0000","last_active":"2024-04-16T15:40:29.919923+0000","last_peered":"2024-04-16T15:40:29.919923+0000","last_clean":"2024-04-16T15:40:29.919923+0000","last_became_active":"2024-04-16T15:40:24.550410+0000","last_became_peered":"2024-04-16T15:40:24.550410+0000","last_unstale":"2024-04-16T15:40:29.919923+0000","last_undegraded":"2024-04-16T15:40:29.919923+0000","last_fullsized":"2024-04-16T15:40:29.919923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:32.578262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567285+0000","last_change":"2024-04-16T15:40:23.531043+0000","last_active":"2024-04-16T15:40:29.567285+0000","last_peered":"2024-04-16T15:40:29.567285+0000","last_clean":"2024-04-16T15:40:29.567285+0000","last_became_active":"2024-04-16T15:40:23.530764+0000","last_became_peered":"2024-04-16T15:40:23.530764+0000","last_unstale":"2024-04-16T15:40:29.567285+0000","last_undegraded":"2024-04-16T15:40:29.567285+0000","last_fullsized":"2024-04-16T15:40:29.567285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566507+0000","last_change":"2024-04-16T15:40:28.805696+0000","last_active":"2024-04-16T15:40:29.566507+0000","last_peered":"2024-04-16T15:40:29.566507+0000","last_clean":"2024-04-16T15:40:29.566507+0000","last_became_active":"2024-04-16T15:40:28.805319+0000","last_became_peered":"2024-04-16T15:40:28.805319+0000","last_unstale":"2024-04-16T15:40:29.566507+0000","last_undegraded":"2024-04-16T15:40:29.566507+0000","last_fullsized":"2024-04-16T15:40:29.566507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:55.036666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918914+0000","last_change":"2024-04-16T15:40:28.804646+0000","last_active":"2024-04-16T15:40:29.918914+0000","last_peered":"2024-04-16T15:40:29.918914+0000","last_clean":"2024-04-16T15:40:29.918914+0000","last_became_active":"2024-04-16T15:40:28.804239+0000","last_became_peered":"2024-04-16T15:40:28.804239+0000","last_unstale":"2024-04-16T15:40:29.918914+0000","last_undegraded":"2024-04-16T15:40:29.918914+0000","last_fullsized":"2024-04-16T15:40:29.918914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:36.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769103+0000","last_change":"2024-04-16T15:40:24.533204+0000","last_active":"2024-04-16T15:40:29.769103+0000","last_peered":"2024-04-16T15:40:29.769103+0000","last_clean":"2024-04-16T15:40:29.769103+0000","last_became_active":"2024-04-16T15:40:24.532947+0000","last_became_peered":"2024-04-16T15:40:24.532947+0000","last_unstale":"2024-04-16T15:40:29.769103+0000","last_undegraded":"2024-04-16T15:40:29.769103+0000","last_fullsized":"2024-04-16T15:40:29.769103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919749+0000","last_change":"2024-04-16T15:40:23.537693+0000","last_active":"2024-04-16T15:40:29.919749+0000","last_peered":"2024-04-16T15:40:29.919749+0000","last_clean":"2024-04-16T15:40:29.919749+0000","last_became_active":"2024-04-16T15:40:23.536396+0000","last_became_peered":"2024-04-16T15:40:23.536396+0000","last_unstale":"2024-04-16T15:40:29.919749+0000","last_undegraded":"2024-04-16T15:40:29.919749+0000","last_fullsized":"2024-04-16T15:40:29.919749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567324+0000","last_change":"2024-04-16T15:40:28.567889+0000","last_active":"2024-04-16T15:40:29.567324+0000","last_peered":"2024-04-16T15:40:29.567324+0000","last_clean":"2024-04-16T15:40:29.567324+0000","last_became_active":"2024-04-16T15:40:28.567519+0000","last_became_peered":"2024-04-16T15:40:28.567519+0000","last_unstale":"2024-04-16T15:40:29.567324+0000","last_undegraded":"2024-04-16T15:40:29.567324+0000","last_fullsized":"2024-04-16T15:40:29.567324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566216+0000","last_change":"2024-04-16T15:40:24.550785+0000","last_active":"2024-04-16T15:40:29.566216+0000","last_peered":"2024-04-16T15:40:29.566216+0000","last_clean":"2024-04-16T15:40:29.566216+0000","last_became_active":"2024-04-16T15:40:24.550492+0000","last_became_peered":"2024-04-16T15:40:24.550492+0000","last_unstale":"2024-04-16T15:40:29.566216+0000","last_undegraded":"2024-04-16T15:40:29.566216+0000","last_fullsized":"2024-04-16T15:40:29.566216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567393+0000","last_change":"2024-04-16T15:40:23.527649+0000","last_active":"2024-04-16T15:40:29.567393+0000","last_peered":"2024-04-16T15:40:29.567393+0000","last_clean":"2024-04-16T15:40:29.567393+0000","last_became_active":"2024-04-16T15:40:23.526805+0000","last_became_peered":"2024-04-16T15:40:23.526805+0000","last_unstale":"2024-04-16T15:40:29.567393+0000","last_undegraded":"2024-04-16T15:40:29.567393+0000","last_fullsized":"2024-04-16T15:40:29.567393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:17.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566914+0000","last_change":"2024-04-16T15:40:23.536732+0000","last_active":"2024-04-16T15:40:29.566914+0000","last_peered":"2024-04-16T15:40:29.566914+0000","last_clean":"2024-04-16T15:40:29.566914+0000","last_became_active":"2024-04-16T15:40:23.536431+0000","last_became_peered":"2024-04-16T15:40:23.536431+0000","last_unstale":"2024-04-16T15:40:29.566914+0000","last_undegraded":"2024-04-16T15:40:29.566914+0000","last_fullsized":"2024-04-16T15:40:29.566914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918955+0000","last_change":"2024-04-16T15:40:24.538554+0000","last_active":"2024-04-16T15:40:29.918955+0000","last_peered":"2024-04-16T15:40:29.918955+0000","last_clean":"2024-04-16T15:40:29.918955+0000","last_became_active":"2024-04-16T15:40:24.538271+0000","last_became_peered":"2024-04-16T15:40:24.538271+0000","last_unstale":"2024-04-16T15:40:29.918955+0000","last_undegraded":"2024-04-16T15:40:29.918955+0000","last_fullsized":"2024-04-16T15:40:29.918955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567207+0000","last_change":"2024-04-16T15:40:28.566595+0000","last_active":"2024-04-16T15:40:29.567207+0000","last_peered":"2024-04-16T15:40:29.567207+0000","last_clean":"2024-04-16T15:40:29.567207+0000","last_became_active":"2024-04-16T15:40:28.566177+0000","last_became_peered":"2024-04-16T15:40:28.566177+0000","last_unstale":"2024-04-16T15:40:29.567207+0000","last_undegraded":"2024-04-16T15:40:29.567207+0000","last_fullsized":"2024-04-16T15:40:29.567207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:34.092734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567347+0000","last_change":"2024-04-16T15:40:23.528855+0000","last_active":"2024-04-16T15:40:29.567347+0000","last_peered":"2024-04-16T15:40:29.567347+0000","last_clean":"2024-04-16T15:40:29.567347+0000","last_became_active":"2024-04-16T15:40:23.528616+0000","last_became_peered":"2024-04-16T15:40:23.528616+0000","last_unstale":"2024-04-16T15:40:29.567347+0000","last_undegraded":"2024-04-16T15:40:29.567347+0000","last_fullsized":"2024-04-16T15:40:29.567347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919604+0000","last_change":"2024-04-16T15:40:24.535332+0000","last_active":"2024-04-16T15:40:29.919604+0000","last_peered":"2024-04-16T15:40:29.919604+0000","last_clean":"2024-04-16T15:40:29.919604+0000","last_became_active":"2024-04-16T15:40:24.535031+0000","last_became_peered":"2024-04-16T15:40:24.535031+0000","last_unstale":"2024-04-16T15:40:29.919604+0000","last_undegraded":"2024-04-16T15:40:29.919604+0000","last_fullsized":"2024-04-16T15:40:29.919604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:45.872510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567748+0000","last_change":"2024-04-16T15:40:28.564309+0000","last_active":"2024-04-16T15:40:29.567748+0000","last_peered":"2024-04-16T15:40:29.567748+0000","last_clean":"2024-04-16T15:40:29.567748+0000","last_became_active":"2024-04-16T15:40:28.563916+0000","last_became_peered":"2024-04-16T15:40:28.563916+0000","last_unstale":"2024-04-16T15:40:29.567748+0000","last_undegraded":"2024-04-16T15:40:29.567748+0000","last_fullsized":"2024-04-16T15:40:29.567748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:02:28.301124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566423+0000","last_change":"2024-04-16T15:40:24.543242+0000","last_active":"2024-04-16T15:40:29.566423+0000","last_peered":"2024-04-16T15:40:29.566423+0000","last_clean":"2024-04-16T15:40:29.566423+0000","last_became_active":"2024-04-16T15:40:24.542947+0000","last_became_peered":"2024-04-16T15:40:24.542947+0000","last_unstale":"2024-04-16T15:40:29.566423+0000","last_undegraded":"2024-04-16T15:40:29.566423+0000","last_fullsized":"2024-04-16T15:40:29.566423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566822+0000","last_change":"2024-04-16T15:40:24.535196+0000","last_active":"2024-04-16T15:40:29.566822+0000","last_peered":"2024-04-16T15:40:29.566822+0000","last_clean":"2024-04-16T15:40:29.566822+0000","last_became_active":"2024-04-16T15:40:24.534885+0000","last_became_peered":"2024-04-16T15:40:24.534885+0000","last_unstale":"2024-04-16T15:40:29.566822+0000","last_undegraded":"2024-04-16T15:40:29.566822+0000","last_fullsized":"2024-04-16T15:40:29.566822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:20.979138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920587+0000","last_change":"2024-04-16T15:40:24.548666+0000","last_active":"2024-04-16T15:40:29.920587+0000","last_peered":"2024-04-16T15:40:29.920587+0000","last_clean":"2024-04-16T15:40:29.920587+0000","last_became_active":"2024-04-16T15:40:24.548388+0000","last_became_peered":"2024-04-16T15:40:24.548388+0000","last_unstale":"2024-04-16T15:40:29.920587+0000","last_undegraded":"2024-04-16T15:40:29.920587+0000","last_fullsized":"2024-04-16T15:40:29.920587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:28.169445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918853+0000","last_change":"2024-04-16T15:40:24.542842+0000","last_active":"2024-04-16T15:40:29.918853+0000","last_peered":"2024-04-16T15:40:29.918853+0000","last_clean":"2024-04-16T15:40:29.918853+0000","last_became_active":"2024-04-16T15:40:24.542514+0000","last_became_peered":"2024-04-16T15:40:24.542514+0000","last_unstale":"2024-04-16T15:40:29.918853+0000","last_undegraded":"2024-04-16T15:40:29.918853+0000","last_fullsized":"2024-04-16T15:40:29.918853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567390+0000","last_change":"2024-04-16T15:40:24.543863+0000","last_active":"2024-04-16T15:40:29.567390+0000","last_peered":"2024-04-16T15:40:29.567390+0000","last_clean":"2024-04-16T15:40:29.567390+0000","last_became_active":"2024-04-16T15:40:24.543611+0000","last_became_peered":"2024-04-16T15:40:24.543611+0000","last_unstale":"2024-04-16T15:40:29.567390+0000","last_undegraded":"2024-04-16T15:40:29.567390+0000","last_fullsized":"2024-04-16T15:40:29.567390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769571+0000","last_change":"2024-04-16T15:40:24.533049+0000","last_active":"2024-04-16T15:40:29.769571+0000","last_peered":"2024-04-16T15:40:29.769571+0000","last_clean":"2024-04-16T15:40:29.769571+0000","last_became_active":"2024-04-16T15:40:24.532782+0000","last_became_peered":"2024-04-16T15:40:24.532782+0000","last_unstale":"2024-04-16T15:40:29.769571+0000","last_undegraded":"2024-04-16T15:40:29.769571+0000","last_fullsized":"2024-04-16T15:40:29.769571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567235+0000","last_change":"2024-04-16T15:40:24.546110+0000","last_active":"2024-04-16T15:40:29.567235+0000","last_peered":"2024-04-16T15:40:29.567235+0000","last_clean":"2024-04-16T15:40:29.567235+0000","last_became_active":"2024-04-16T15:40:24.545660+0000","last_became_peered":"2024-04-16T15:40:24.545660+0000","last_unstale":"2024-04-16T15:40:29.567235+0000","last_undegraded":"2024-04-16T15:40:29.567235+0000","last_fullsized":"2024-04-16T15:40:29.567235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567701+0000","last_change":"2024-04-16T15:40:24.543839+0000","last_active":"2024-04-16T15:40:29.567701+0000","last_peered":"2024-04-16T15:40:29.567701+0000","last_clean":"2024-04-16T15:40:29.567701+0000","last_became_active":"2024-04-16T15:40:24.543560+0000","last_became_peered":"2024-04-16T15:40:24.543560+0000","last_unstale":"2024-04-16T15:40:29.567701+0000","last_undegraded":"2024-04-16T15:40:29.567701+0000","last_fullsized":"2024-04-16T15:40:29.567701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919650+0000","last_change":"2024-04-16T15:40:24.533578+0000","last_active":"2024-04-16T15:40:29.919650+0000","last_peered":"2024-04-16T15:40:29.919650+0000","last_clean":"2024-04-16T15:40:29.919650+0000","last_became_active":"2024-04-16T15:40:24.533335+0000","last_became_peered":"2024-04-16T15:40:24.533335+0000","last_unstale":"2024-04-16T15:40:29.919650+0000","last_undegraded":"2024-04-16T15:40:29.919650+0000","last_fullsized":"2024-04-16T15:40:29.919650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567192+0000","last_change":"2024-04-16T15:40:24.545340+0000","last_active":"2024-04-16T15:40:29.567192+0000","last_peered":"2024-04-16T15:40:29.567192+0000","last_clean":"2024-04-16T15:40:29.567192+0000","last_became_active":"2024-04-16T15:40:24.545096+0000","last_became_peered":"2024-04-16T15:40:24.545096+0000","last_unstale":"2024-04-16T15:40:29.567192+0000","last_undegraded":"2024-04-16T15:40:29.567192+0000","last_fullsized":"2024-04-16T15:40:29.567192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920042+0000","last_change":"2024-04-16T15:40:24.551716+0000","last_active":"2024-04-16T15:40:29.920042+0000","last_peered":"2024-04-16T15:40:29.920042+0000","last_clean":"2024-04-16T15:40:29.920042+0000","last_became_active":"2024-04-16T15:40:24.551360+0000","last_became_peered":"2024-04-16T15:40:24.551360+0000","last_unstale":"2024-04-16T15:40:29.920042+0000","last_undegraded":"2024-04-16T15:40:29.920042+0000","last_fullsized":"2024-04-16T15:40:29.920042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:14.076265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918705+0000","last_change":"2024-04-16T15:40:24.549554+0000","last_active":"2024-04-16T15:40:29.918705+0000","last_peered":"2024-04-16T15:40:29.918705+0000","last_clean":"2024-04-16T15:40:29.918705+0000","last_became_active":"2024-04-16T15:40:24.548848+0000","last_became_peered":"2024-04-16T15:40:24.548848+0000","last_unstale":"2024-04-16T15:40:29.918705+0000","last_undegraded":"2024-04-16T15:40:29.918705+0000","last_fullsized":"2024-04-16T15:40:29.918705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567025+0000","last_change":"2024-04-16T15:40:24.539612+0000","last_active":"2024-04-16T15:40:29.567025+0000","last_peered":"2024-04-16T15:40:29.567025+0000","last_clean":"2024-04-16T15:40:29.567025+0000","last_became_active":"2024-04-16T15:40:24.536911+0000","last_became_peered":"2024-04-16T15:40:24.536911+0000","last_unstale":"2024-04-16T15:40:29.567025+0000","last_undegraded":"2024-04-16T15:40:29.567025+0000","last_fullsized":"2024-04-16T15:40:29.567025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921716+0000","last_change":"2024-04-16T15:40:24.550620+0000","last_active":"2024-04-16T15:40:29.921716+0000","last_peered":"2024-04-16T15:40:29.921716+0000","last_clean":"2024-04-16T15:40:29.921716+0000","last_became_active":"2024-04-16T15:40:24.549587+0000","last_became_peered":"2024-04-16T15:40:24.549587+0000","last_unstale":"2024-04-16T15:40:29.921716+0000","last_undegraded":"2024-04-16T15:40:29.921716+0000","last_fullsized":"2024-04-16T15:40:29.921716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:27.826954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566610+0000","last_change":"2024-04-16T15:40:24.539758+0000","last_active":"2024-04-16T15:40:29.566610+0000","last_peered":"2024-04-16T15:40:29.566610+0000","last_clean":"2024-04-16T15:40:29.566610+0000","last_became_active":"2024-04-16T15:40:24.539525+0000","last_became_peered":"2024-04-16T15:40:24.539525+0000","last_unstale":"2024-04-16T15:40:29.566610+0000","last_undegraded":"2024-04-16T15:40:29.566610+0000","last_fullsized":"2024-04-16T15:40:29.566610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:44.286135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921033+0000","last_change":"2024-04-16T15:40:24.548867+0000","last_active":"2024-04-16T15:40:29.921033+0000","last_peered":"2024-04-16T15:40:29.921033+0000","last_clean":"2024-04-16T15:40:29.921033+0000","last_became_active":"2024-04-16T15:40:24.548585+0000","last_became_peered":"2024-04-16T15:40:24.548585+0000","last_unstale":"2024-04-16T15:40:29.921033+0000","last_undegraded":"2024-04-16T15:40:29.921033+0000","last_fullsized":"2024-04-16T15:40:29.921033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:31.030716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919546+0000","last_change":"2024-04-16T15:40:24.545697+0000","last_active":"2024-04-16T15:40:29.919546+0000","last_peered":"2024-04-16T15:40:29.919546+0000","last_clean":"2024-04-16T15:40:29.919546+0000","last_became_active":"2024-04-16T15:40:24.545409+0000","last_became_peered":"2024-04-16T15:40:24.545409+0000","last_unstale":"2024-04-16T15:40:29.919546+0000","last_undegraded":"2024-04-16T15:40:29.919546+0000","last_fullsized":"2024-04-16T15:40:29.919546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769173+0000","last_change":"2024-04-16T15:40:24.544806+0000","last_active":"2024-04-16T15:40:29.769173+0000","last_peered":"2024-04-16T15:40:29.769173+0000","last_clean":"2024-04-16T15:40:29.769173+0000","last_became_active":"2024-04-16T15:40:24.544550+0000","last_became_peered":"2024-04-16T15:40:24.544550+0000","last_unstale":"2024-04-16T15:40:29.769173+0000","last_undegraded":"2024-04-16T15:40:29.769173+0000","last_fullsized":"2024-04-16T15:40:29.769173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918672+0000","last_change":"2024-04-16T15:40:24.542813+0000","last_active":"2024-04-16T15:40:29.918672+0000","last_peered":"2024-04-16T15:40:29.918672+0000","last_clean":"2024-04-16T15:40:29.918672+0000","last_became_active":"2024-04-16T15:40:24.534241+0000","last_became_peered":"2024-04-16T15:40:24.534241+0000","last_unstale":"2024-04-16T15:40:29.918672+0000","last_undegraded":"2024-04-16T15:40:29.918672+0000","last_fullsized":"2024-04-16T15:40:29.918672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:30.065862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567152+0000","last_change":"2024-04-16T15:40:24.546408+0000","last_active":"2024-04-16T15:40:29.567152+0000","last_peered":"2024-04-16T15:40:29.567152+0000","last_clean":"2024-04-16T15:40:29.567152+0000","last_became_active":"2024-04-16T15:40:24.546168+0000","last_became_peered":"2024-04-16T15:40:24.546168+0000","last_unstale":"2024-04-16T15:40:29.567152+0000","last_undegraded":"2024-04-16T15:40:29.567152+0000","last_fullsized":"2024-04-16T15:40:29.567152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920159+0000","last_change":"2024-04-16T15:40:24.533057+0000","last_active":"2024-04-16T15:40:29.920159+0000","last_peered":"2024-04-16T15:40:29.920159+0000","last_clean":"2024-04-16T15:40:29.920159+0000","last_became_active":"2024-04-16T15:40:24.532732+0000","last_became_peered":"2024-04-16T15:40:24.532732+0000","last_unstale":"2024-04-16T15:40:29.920159+0000","last_undegraded":"2024-04-16T15:40:29.920159+0000","last_fullsized":"2024-04-16T15:40:29.920159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:13.766871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567143+0000","last_change":"2024-04-16T15:40:24.540869+0000","last_active":"2024-04-16T15:40:29.567143+0000","last_peered":"2024-04-16T15:40:29.567143+0000","last_clean":"2024-04-16T15:40:29.567143+0000","last_became_active":"2024-04-16T15:40:24.537498+0000","last_became_peered":"2024-04-16T15:40:24.537498+0000","last_unstale":"2024-04-16T15:40:29.567143+0000","last_undegraded":"2024-04-16T15:40:29.567143+0000","last_fullsized":"2024-04-16T15:40:29.567143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567216+0000","last_change":"2024-04-16T15:40:24.536207+0000","last_active":"2024-04-16T15:40:29.567216+0000","last_peered":"2024-04-16T15:40:29.567216+0000","last_clean":"2024-04-16T15:40:29.567216+0000","last_became_active":"2024-04-16T15:40:24.535940+0000","last_became_peered":"2024-04-16T15:40:24.535940+0000","last_unstale":"2024-04-16T15:40:29.567216+0000","last_undegraded":"2024-04-16T15:40:29.567216+0000","last_fullsized":"2024-04-16T15:40:29.567216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920836+0000","last_change":"2024-04-16T15:40:24.544527+0000","last_active":"2024-04-16T15:40:29.920836+0000","last_peered":"2024-04-16T15:40:29.920836+0000","last_clean":"2024-04-16T15:40:29.920836+0000","last_became_active":"2024-04-16T15:40:24.543702+0000","last_became_peered":"2024-04-16T15:40:24.543702+0000","last_unstale":"2024-04-16T15:40:29.920836+0000","last_undegraded":"2024-04-16T15:40:29.920836+0000","last_fullsized":"2024-04-16T15:40:29.920836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:24.380253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566076+0000","last_change":"2024-04-16T15:40:24.545972+0000","last_active":"2024-04-16T15:40:29.566076+0000","last_peered":"2024-04-16T15:40:29.566076+0000","last_clean":"2024-04-16T15:40:29.566076+0000","last_became_active":"2024-04-16T15:40:24.545726+0000","last_became_peered":"2024-04-16T15:40:24.545726+0000","last_unstale":"2024-04-16T15:40:29.566076+0000","last_undegraded":"2024-04-16T15:40:29.566076+0000","last_fullsized":"2024-04-16T15:40:29.566076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919710+0000","last_change":"2024-04-16T15:40:24.550333+0000","last_active":"2024-04-16T15:40:29.919710+0000","last_peered":"2024-04-16T15:40:29.919710+0000","last_clean":"2024-04-16T15:40:29.919710+0000","last_became_active":"2024-04-16T15:40:24.550063+0000","last_became_peered":"2024-04-16T15:40:24.550063+0000","last_unstale":"2024-04-16T15:40:29.919710+0000","last_undegraded":"2024-04-16T15:40:29.919710+0000","last_fullsized":"2024-04-16T15:40:29.919710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920040+0000","last_change":"2024-04-16T15:40:24.545717+0000","last_active":"2024-04-16T15:40:29.920040+0000","last_peered":"2024-04-16T15:40:29.920040+0000","last_clean":"2024-04-16T15:40:29.920040+0000","last_became_active":"2024-04-16T15:40:24.545445+0000","last_became_peered":"2024-04-16T15:40:24.545445+0000","last_unstale":"2024-04-16T15:40:29.920040+0000","last_undegraded":"2024-04-16T15:40:29.920040+0000","last_fullsized":"2024-04-16T15:40:29.920040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919205+0000","last_change":"2024-04-16T15:40:24.549202+0000","last_active":"2024-04-16T15:40:29.919205+0000","last_peered":"2024-04-16T15:40:29.919205+0000","last_clean":"2024-04-16T15:40:29.919205+0000","last_became_active":"2024-04-16T15:40:24.548794+0000","last_became_peered":"2024-04-16T15:40:24.548794+0000","last_unstale":"2024-04-16T15:40:29.919205+0000","last_undegraded":"2024-04-16T15:40:29.919205+0000","last_fullsized":"2024-04-16T15:40:29.919205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:25.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919742+0000","last_change":"2024-04-16T15:40:24.548634+0000","last_active":"2024-04-16T15:40:29.919742+0000","last_peered":"2024-04-16T15:40:29.919742+0000","last_clean":"2024-04-16T15:40:29.919742+0000","last_became_active":"2024-04-16T15:40:24.548342+0000","last_became_peered":"2024-04-16T15:40:24.548342+0000","last_unstale":"2024-04-16T15:40:29.919742+0000","last_undegraded":"2024-04-16T15:40:29.919742+0000","last_fullsized":"2024-04-16T15:40:29.919742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:18.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918832+0000","last_change":"2024-04-16T15:40:24.534160+0000","last_active":"2024-04-16T15:40:29.918832+0000","last_peered":"2024-04-16T15:40:29.918832+0000","last_clean":"2024-04-16T15:40:29.918832+0000","last_became_active":"2024-04-16T15:40:24.533913+0000","last_became_peered":"2024-04-16T15:40:24.533913+0000","last_unstale":"2024-04-16T15:40:29.918832+0000","last_undegraded":"2024-04-16T15:40:29.918832+0000","last_fullsized":"2024-04-16T15:40:29.918832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769869+0000","last_change":"2024-04-16T15:40:24.546471+0000","last_active":"2024-04-16T15:40:29.769869+0000","last_peered":"2024-04-16T15:40:29.769869+0000","last_clean":"2024-04-16T15:40:29.769869+0000","last_became_active":"2024-04-16T15:40:24.545638+0000","last_became_peered":"2024-04-16T15:40:24.545638+0000","last_unstale":"2024-04-16T15:40:29.769869+0000","last_undegraded":"2024-04-16T15:40:29.769869+0000","last_fullsized":"2024-04-16T15:40:29.769869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920784+0000","last_change":"2024-04-16T15:40:24.546172+0000","last_active":"2024-04-16T15:40:29.920784+0000","last_peered":"2024-04-16T15:40:29.920784+0000","last_clean":"2024-04-16T15:40:29.920784+0000","last_became_active":"2024-04-16T15:40:24.545582+0000","last_became_peered":"2024-04-16T15:40:24.545582+0000","last_unstale":"2024-04-16T15:40:29.920784+0000","last_undegraded":"2024-04-16T15:40:29.920784+0000","last_fullsized":"2024-04-16T15:40:29.920784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:21.801823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:40:42.098 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-16T15:40:42.098 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-16T15:40:42.098 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-16T15:40:42.099 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-16T15:40:42.099 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-16T15:40:42.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.237+0000 7fb067b59640 1 Processor -- start 2024-04-16T15:40:42.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.237+0000 7fb067b59640 1 -- start start 2024-04-16T15:40:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.237+0000 7fb067b59640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb060159520 0x7fb060159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.237+0000 7fb067b59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 0x7fb06006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.237+0000 7fb067b59640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb06006e470 0x7fb0600ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb067b59640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fb0600c9540 con 0x7fb06006e470 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb067b59640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb0600ae4b0 con 0x7fb060159520 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb067b59640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb0600c6940 con 0x7fb0600c78d0 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0660f6640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb06006e470 0x7fb0600ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0660f6640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb06006e470 0x7fb0600ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46762/0 (socket says 172.21.15.71:46762) 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0660f6640 1 -- 172.21.15.71:0/3631031791 learned_addr learned my addr 172.21.15.71:0/3631031791 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0658f5640 1 --2- 172.21.15.71:0/3631031791 >> v2:172.21.15.122:3300/0 conn(0x7fb060159520 0x7fb060159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0660f6640 1 -- 172.21.15.71:0/3631031791 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 msgr2=0x7fb06006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0650f4640 1 --2- 172.21.15.71:0/3631031791 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 0x7fb06006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0660f6640 1 --2- 172.21.15.71:0/3631031791 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 0x7fb06006df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0660f6640 1 -- 172.21.15.71:0/3631031791 >> v2:172.21.15.122:3300/0 conn(0x7fb060159520 msgr2=0x7fb060159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0660f6640 1 --2- 172.21.15.71:0/3631031791 >> v2:172.21.15.122:3300/0 conn(0x7fb060159520 0x7fb060159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.238+0000 7fb0660f6640 1 -- 172.21.15.71:0/3631031791 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06015a030 con 0x7fb06006e470 2024-04-16T15:40:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb0660f6640 1 --2- 172.21.15.71:0/3631031791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ca270 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb0600cb020 tx=0x7fb05802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21505e63c1c7295 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb0648f3640 1 -- 172.21.15.71:0/3631031791 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058038070 con 0x7fb06006e470 2024-04-16T15:40:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb0648f3640 1 -- 172.21.15.71:0/3631031791 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb05802f050 con 0x7fb06006e470 2024-04-16T15:40:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb0648f3640 1 -- 172.21.15.71:0/3631031791 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058031970 con 0x7fb06006e470 2024-04-16T15:40:42.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb067b59640 1 -- 172.21.15.71:0/3631031791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 msgr2=0x7fb0600ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb067b59640 1 --2- 172.21.15.71:0/3631031791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ca270 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb0600cb020 tx=0x7fb05802b7f0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb067b59640 1 -- 172.21.15.71:0/3631031791 shutdown_connections 2024-04-16T15:40:42.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb067b59640 1 --2- 172.21.15.71:0/3631031791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ca270 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb067b59640 1 --2- 172.21.15.71:0/3631031791 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 0x7fb06006df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb067b59640 1 --2- 172.21.15.71:0/3631031791 >> v2:172.21.15.122:3300/0 conn(0x7fb060159520 0x7fb060159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.239+0000 7fb067b59640 1 -- 172.21.15.71:0/3631031791 >> 172.21.15.71:0/3631031791 conn(0x7fb0600cb820 msgr2=0x7fb0600cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.240+0000 7fb067b59640 1 -- 172.21.15.71:0/3631031791 shutdown_connections 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.240+0000 7fb067b59640 1 -- 172.21.15.71:0/3631031791 wait complete. 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb067b59640 1 Processor -- start 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb067b59640 1 -- start start 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb067b59640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb067b59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 0x7fb0600bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb067b59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb060159520 0x7fb0600c0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb067b59640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0600c9d50 con 0x7fb06006e470 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb067b59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0600c9a70 con 0x7fb060159520 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb067b59640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb0600b0cf0 con 0x7fb0600c78d0 2024-04-16T15:40:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb0658f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb0658f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ac410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46772/0 (socket says 172.21.15.71:46772) 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.241+0000 7fb0658f5640 1 -- 172.21.15.71:0/154935765 learned_addr learned my addr 172.21.15.71:0/154935765 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0650f4640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 0x7fb0600bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0660f6640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb060159520 0x7fb0600c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0658f5640 1 -- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 msgr2=0x7fb0600bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0658f5640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 0x7fb0600bfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0658f5640 1 -- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb060159520 msgr2=0x7fb0600c0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0658f5640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb060159520 0x7fb0600c0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0658f5640 1 -- 172.21.15.71:0/154935765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0601566e0 con 0x7fb06006e470 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0660f6640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb060159520 0x7fb0600c0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb0658f5640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ac410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb05000cd00 tx=0x7fb05000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb05e7fc640 1 -- 172.21.15.71:0/154935765 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050003990 con 0x7fb06006e470 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb05e7fc640 1 -- 172.21.15.71:0/154935765 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb050003b30 con 0x7fb06006e470 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb060154c20 con 0x7fb06006e470 2024-04-16T15:40:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb05e7fc640 1 -- 172.21.15.71:0/154935765 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050004020 con 0x7fb06006e470 2024-04-16T15:40:42.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.242+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0600cad20 con 0x7fb06006e470 2024-04-16T15:40:42.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.243+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb060074060 con 0x7fb06006e470 2024-04-16T15:40:42.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.243+0000 7fb05e7fc640 1 -- 172.21.15.71:0/154935765 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0500041c0 con 0x7fb06006e470 2024-04-16T15:40:42.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.244+0000 7fb05e7fc640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb04c06c920 0x7fb04c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.244+0000 7fb0650f4640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb04c06c920 0x7fb04c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.244+0000 7fb05e7fc640 1 -- 172.21.15.71:0/154935765 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb05008c2e0 con 0x7fb06006e470 2024-04-16T15:40:42.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.244+0000 7fb0650f4640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb04c06c920 0x7fb04c06ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb060156b20 tx=0x7fb054008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.248+0000 7fb05e7fc640 1 -- 172.21.15.71:0/154935765 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb050014030 con 0x7fb06006e470 2024-04-16T15:40:42.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.473+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb0600bebc0 con 0x7fb06006e470 2024-04-16T15:40:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.476+0000 7fb05e7fc640 1 -- 172.21.15.71:0/154935765 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174136 (secure 0 0 0) 0x7fb05005a460 con 0x7fb06006e470 2024-04-16T15:40:42.477 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb04c06c920 msgr2=0x7fb04c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb04c06c920 0x7fb04c06ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb060156b20 tx=0x7fb054008040 comp rx=0 tx=0).stop 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 msgr2=0x7fb0600ac410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ac410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb05000cd00 tx=0x7fb05000a5c0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 shutdown_connections 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb04c06c920 0x7fb04c06ede0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb060159520 0x7fb0600c0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0600c78d0 0x7fb0600bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 --2- 172.21.15.71:0/154935765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb06006e470 0x7fb0600ac410 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 >> 172.21.15.71:0/154935765 conn(0x7fb0600cb820 msgr2=0x7fb06006c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 shutdown_connections 2024-04-16T15:40:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.478+0000 7fb067b59640 1 -- 172.21.15.71:0/154935765 wait complete. 2024-04-16T15:40:42.490 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4207135910},{"type":"v1","addr":"172.21.15.71:6833","nonce":4207135910}]},"active_addr":"172.21.15.71:6833/4207135910","active_change":"2024-04-16T15:40:03.686041+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":868772958}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":2409354177}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":3649889507}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":3788844221}]}]} 2024-04-16T15:40:42.494 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-16T15:40:42.494 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-16T15:40:42.494 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:42.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.597+0000 7fdfbec88640 1 Processor -- start 2024-04-16T15:40:42.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.597+0000 7fdfbec88640 1 -- start start 2024-04-16T15:40:42.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.597+0000 7fdfbec88640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fdfb8153210 0x7fdfb80aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbec88640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdfb806fb20 0x7fdfb806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbec88640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb80ab030 con 0x7fdfb8156620 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbec88640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb8070490 con 0x7fdfb806fb20 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbec88640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb81535f0 con 0x7fdfb8153210 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbd225640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdfb806fb20 0x7fdfb806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfb7fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fdfb8156620 0x7fdfb806da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34172/0 (socket says 172.21.15.71:34172) 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfb7fff640 1 -- 172.21.15.71:0/3403953382 learned_addr learned my addr 172.21.15.71:0/3403953382 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbca24640 1 --2- 172.21.15.71:0/3403953382 >> v2:172.21.15.122:3300/0 conn(0x7fdfb8153210 0x7fdfb80aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbd225640 1 -- 172.21.15.71:0/3403953382 >> v2:172.21.15.122:3300/0 conn(0x7fdfb8153210 msgr2=0x7fdfb80aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbd225640 1 --2- 172.21.15.71:0/3403953382 >> v2:172.21.15.122:3300/0 conn(0x7fdfb8153210 0x7fdfb80aa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbd225640 1 -- 172.21.15.71:0/3403953382 >> v1:172.21.15.71:6789/0 conn(0x7fdfb8156620 legacy=0x7fdfb806da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfbd225640 1 -- 172.21.15.71:0/3403953382 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb80c1c10 con 0x7fdfb806fb20 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.598+0000 7fdfb77fe640 1 -- 172.21.15.71:0/3403953382 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1680706151 0 0) 0x7fdfb80ab030 con 0x7fdfb8156620 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfbd225640 1 --2- 172.21.15.71:0/3403953382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb806ff50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80c5f30 tx=0x7fdfb002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9909427379b0b868 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfb77fe640 1 -- 172.21.15.71:0/3403953382 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0039070 con 0x7fdfb806fb20 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfb77fe640 1 -- 172.21.15.71:0/3403953382 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb002cbf0 con 0x7fdfb806fb20 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfb77fe640 1 -- 172.21.15.71:0/3403953382 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0034440 con 0x7fdfb806fb20 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfbec88640 1 -- 172.21.15.71:0/3403953382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 msgr2=0x7fdfb806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3403953382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb806ff50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80c5f30 tx=0x7fdfb002be40 comp rx=0 tx=0).stop 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfbec88640 1 -- 172.21.15.71:0/3403953382 shutdown_connections 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3403953382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb806ff50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3403953382 >> v2:172.21.15.122:3300/0 conn(0x7fdfb8153210 0x7fdfb80aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.599+0000 7fdfbec88640 1 -- 172.21.15.71:0/3403953382 >> 172.21.15.71:0/3403953382 conn(0x7fdfb8072aa0 msgr2=0x7fdfb8072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:42.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.600+0000 7fdfbec88640 1 -- 172.21.15.71:0/3403953382 shutdown_connections 2024-04-16T15:40:42.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.600+0000 7fdfbec88640 1 -- 172.21.15.71:0/3403953382 wait complete. 2024-04-16T15:40:42.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.601+0000 7fdfbec88640 1 Processor -- start 2024-04-16T15:40:42.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.601+0000 7fdfbec88640 1 -- start start 2024-04-16T15:40:42.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.601+0000 7fdfbec88640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb81ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.601+0000 7fdfbec88640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfb8153210 0x7fdfb81ef610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.601+0000 7fdfbec88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb8156620 0x7fdfb814b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.601+0000 7fdfbec88640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8156ab0 con 0x7fdfb806fb20 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.601+0000 7fdfbec88640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8070dd0 con 0x7fdfb8156620 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.601+0000 7fdfbec88640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb806e160 con 0x7fdfb8153210 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb81ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb81ef0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46784/0 (socket says 172.21.15.71:46784) 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 -- 172.21.15.71:0/3599541347 learned_addr learned my addr 172.21.15.71:0/3599541347 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbd225640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb8156620 0x7fdfb814b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfb7fff640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfb8153210 0x7fdfb81ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 -- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfb8153210 msgr2=0x7fdfb81ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfb8153210 0x7fdfb81ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 -- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb8156620 msgr2=0x7fdfb814b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb8156620 0x7fdfb814b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 -- 172.21.15.71:0/3599541347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb80c7da0 con 0x7fdfb806fb20 2024-04-16T15:40:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbd225640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb8156620 0x7fdfb814b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfb7fff640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfb8153210 0x7fdfb81ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbca24640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb81ef0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d830 tx=0x7fdfac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfb57fa640 1 -- 172.21.15.71:0/3599541347 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00b6c0 con 0x7fdfb806fb20 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfb57fa640 1 -- 172.21.15.71:0/3599541347 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfac019070 con 0x7fdfb806fb20 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb80c8080 con 0x7fdfb806fb20 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfb57fa640 1 -- 172.21.15.71:0/3599541347 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00ba80 con 0x7fdfb806fb20 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.602+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb80c7380 con 0x7fdfb806fb20 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.603+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb8068d20 con 0x7fdfb806fb20 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.604+0000 7fdfb57fa640 1 -- 172.21.15.71:0/3599541347 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfac00bc20 con 0x7fdfb806fb20 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.604+0000 7fdfb57fa640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf9c06c920 0x7fdf9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.604+0000 7fdfb7fff640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf9c06c920 0x7fdf9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.604+0000 7fdfb57fa640 1 -- 172.21.15.71:0/3599541347 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdfac08c240 con 0x7fdfb806fb20 2024-04-16T15:40:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.605+0000 7fdfb7fff640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf9c06c920 0x7fdf9c06ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80c6970 tx=0x7fdfa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.609+0000 7fdfb57fa640 1 -- 172.21.15.71:0/3599541347 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdfac01c360 con 0x7fdfb806fb20 2024-04-16T15:40:42.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.787+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfb80ae510 con 0x7fdfb806fb20 2024-04-16T15:40:42.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.788+0000 7fdfb57fa640 1 -- 172.21.15.71:0/3599541347 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7fdfac083e20 con 0x7fdfb806fb20 2024-04-16T15:40:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:42.788 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf9c06c920 msgr2=0x7fdf9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf9c06c920 0x7fdf9c06ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80c6970 tx=0x7fdfa4009040 comp rx=0 tx=0).stop 2024-04-16T15:40:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 msgr2=0x7fdfb81ef0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb81ef0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d830 tx=0x7fdfac00dcf0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 shutdown_connections 2024-04-16T15:40:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdf9c06c920 0x7fdf9c06ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfb8156620 0x7fdfb814b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdfb8153210 0x7fdfb81ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 --2- 172.21.15.71:0/3599541347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfb806fb20 0x7fdfb81ef0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 >> 172.21.15.71:0/3599541347 conn(0x7fdfb8072aa0 msgr2=0x7fdfb80c3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 shutdown_connections 2024-04-16T15:40:42.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.790+0000 7fdfbec88640 1 -- 172.21.15.71:0/3599541347 wait complete. 2024-04-16T15:40:42.801 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-16T15:40:42.802 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:42.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.904+0000 7f6294c42640 1 Processor -- start 2024-04-16T15:40:42.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.904+0000 7f6294c42640 1 -- start start 2024-04-16T15:40:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f6294c42640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f629006fc60 0x7f6290154380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f6294c42640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f62900c72e0 0x7f62900c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f6294c42640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6290071750 con 0x7f6290151060 2024-04-16T15:40:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f6294c42640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f6290154990 con 0x7f629006fc60 2024-04-16T15:40:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f6294c42640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6290071520 con 0x7f62900c72e0 2024-04-16T15:40:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f628dd9b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f629006fc60 0x7f6290154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f628e59c640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f6290151060 0x7f62900c4420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34176/0 (socket says 172.21.15.71:34176) 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f628e59c640 1 -- 172.21.15.71:0/256563427 learned_addr learned my addr 172.21.15.71:0/256563427 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.905+0000 7f628ed9d640 1 --2- 172.21.15.71:0/256563427 >> v2:172.21.15.122:3300/0 conn(0x7f62900c72e0 0x7f62900c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.906+0000 7f628dd9b640 1 -- 172.21.15.71:0/256563427 >> v2:172.21.15.122:3300/0 conn(0x7f62900c72e0 msgr2=0x7f62900c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.906+0000 7f628dd9b640 1 --2- 172.21.15.71:0/256563427 >> v2:172.21.15.122:3300/0 conn(0x7f62900c72e0 0x7f62900c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.906+0000 7f628dd9b640 1 -- 172.21.15.71:0/256563427 >> v1:172.21.15.71:6789/0 conn(0x7f6290151060 legacy=0x7f62900c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.906+0000 7f628dd9b640 1 -- 172.21.15.71:0/256563427 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62900719f0 con 0x7f629006fc60 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.906+0000 7f628d59a640 1 -- 172.21.15.71:0/256563427 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722051537 0 0) 0x7f6290071750 con 0x7f6290151060 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.906+0000 7f628dd9b640 1 --2- 172.21.15.71:0/256563427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629006fc60 0x7f6290154380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6290074950 tx=0x7f628402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deee0e634230886c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f628d59a640 1 -- 172.21.15.71:0/256563427 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284035020 con 0x7f629006fc60 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f628d59a640 1 -- 172.21.15.71:0/256563427 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6284031830 con 0x7f629006fc60 2024-04-16T15:40:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f628d59a640 1 -- 172.21.15.71:0/256563427 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284031c00 con 0x7f629006fc60 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f6294c42640 1 -- 172.21.15.71:0/256563427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629006fc60 msgr2=0x7f6290154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f6294c42640 1 --2- 172.21.15.71:0/256563427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629006fc60 0x7f6290154380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6290074950 tx=0x7f628402b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f6294c42640 1 -- 172.21.15.71:0/256563427 shutdown_connections 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f6294c42640 1 --2- 172.21.15.71:0/256563427 >> v2:172.21.15.122:3300/0 conn(0x7f62900c72e0 0x7f62900c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f6294c42640 1 --2- 172.21.15.71:0/256563427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629006fc60 0x7f6290154380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f6294c42640 1 -- 172.21.15.71:0/256563427 >> 172.21.15.71:0/256563427 conn(0x7f62900c93e0 msgr2=0x7f62900c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f6294c42640 1 -- 172.21.15.71:0/256563427 shutdown_connections 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.907+0000 7f6294c42640 1 -- 172.21.15.71:0/256563427 wait complete. 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.908+0000 7f6294c42640 1 Processor -- start 2024-04-16T15:40:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f6294c42640 1 -- start start 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f6294c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f629006fc60 0x7f62901517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f6294c42640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62900c72e0 0x7f62901e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f6294c42640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6290151060 0x7f62900c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f6294c42640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62900c4b30 con 0x7f6290151060 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f6294c42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62900c5260 con 0x7f629006fc60 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f6294c42640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f62900c4cb0 con 0x7f62900c72e0 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f628ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6290151060 0x7f62900c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f628ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6290151060 0x7f62900c2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46802/0 (socket says 172.21.15.71:46802) 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f628ed9d640 1 -- 172.21.15.71:0/356893681 learned_addr learned my addr 172.21.15.71:0/356893681 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f628dd9b640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62900c72e0 0x7f62901e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f628ed9d640 1 -- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62900c72e0 msgr2=0x7f62901e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.909+0000 7f628e59c640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f629006fc60 0x7f62901517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f628ed9d640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62900c72e0 0x7f62901e85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f628ed9d640 1 -- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f629006fc60 msgr2=0x7f62901517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f628ed9d640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f629006fc60 0x7f62901517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f628ed9d640 1 -- 172.21.15.71:0/356893681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62900c8090 con 0x7f6290151060 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f628dd9b640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62900c72e0 0x7f62901e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f628e59c640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f629006fc60 0x7f62901517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f628ed9d640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6290151060 0x7f62900c2810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f628800d830 tx=0x7f628800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f627effd640 1 -- 172.21.15.71:0/356893681 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628800b6c0 con 0x7f6290151060 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f627effd640 1 -- 172.21.15.71:0/356893681 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6288019070 con 0x7f6290151060 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f627effd640 1 -- 172.21.15.71:0/356893681 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628800bac0 con 0x7f6290151060 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6290145f30 con 0x7f6290151060 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.910+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62901464d0 con 0x7f6290151060 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.911+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6290068cf0 con 0x7f6290151060 2024-04-16T15:40:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.911+0000 7f627effd640 1 -- 172.21.15.71:0/356893681 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f628801e460 con 0x7f6290151060 2024-04-16T15:40:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.912+0000 7f627effd640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f627006c8f0 0x7f627006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.912+0000 7f627effd640 1 -- 172.21.15.71:0/356893681 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f628808c4b0 con 0x7f6290151060 2024-04-16T15:40:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.912+0000 7f628e59c640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f627006c8f0 0x7f627006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.912+0000 7f628e59c640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f627006c8f0 0x7f627006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6280007fd0 tx=0x7f6280009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:42.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:42.916+0000 7f627effd640 1 -- 172.21.15.71:0/356893681 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f628805a630 con 0x7f6290151060 2024-04-16T15:40:43.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.091+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62900c87b0 con 0x7f6290151060 2024-04-16T15:40:43.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.092+0000 7f627effd640 1 -- 172.21.15.71:0/356893681 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f6288059f30 con 0x7f6290151060 2024-04-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f627006c8f0 msgr2=0x7f627006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f627006c8f0 0x7f627006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6280007fd0 tx=0x7f6280009040 comp rx=0 tx=0).stop 2024-04-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6290151060 msgr2=0x7f62900c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6290151060 0x7f62900c2810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f628800d830 tx=0x7f628800dcf0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 shutdown_connections 2024-04-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f627006c8f0 0x7f627006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6290151060 0x7f62900c2810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62900c72e0 0x7f62901e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 --2- 172.21.15.71:0/356893681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f629006fc60 0x7f62901517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 >> 172.21.15.71:0/356893681 conn(0x7f62900c93e0 msgr2=0x7f62900c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 shutdown_connections 2024-04-16T15:40:43.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.094+0000 7f6294c42640 1 -- 172.21.15.71:0/356893681 wait complete. 2024-04-16T15:40:43.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-16T15:40:43.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-16T15:40:43.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-16T15:40:43.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-16T15:40:43.109 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-16T15:40:43.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-16T15:40:43.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-16T15:40:43.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-16T15:40:43.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.287+0000 7f211eda8640 1 Processor -- start 2024-04-16T15:40:43.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.287+0000 7fbb01c7e640 1 Processor -- start 2024-04-16T15:40:43.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.287+0000 7f83adf59640 1 Processor -- start 2024-04-16T15:40:43.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.287+0000 7f83adf59640 1 -- start start 2024-04-16T15:40:43.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7f211eda8640 1 -- start start 2024-04-16T15:40:43.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f211eda8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f211814be70 0x7f21180c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7fc84b352640 1 Processor -- start 2024-04-16T15:40:43.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7fc84b352640 1 -- start start 2024-04-16T15:40:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.287+0000 7fbb01c7e640 1 -- start start 2024-04-16T15:40:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7f83adf59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a81531c0 0x7f83a80c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7f83adf59640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f83a80beb50 0x7f83a8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83adf59640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a81535a0 con 0x7f83a80c93c0 2024-04-16T15:40:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83adf59640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f83a80cb7b0 con 0x7f83a80beb50 2024-04-16T15:40:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83adf59640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f83a8156790 con 0x7f83a81531c0 2024-04-16T15:40:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83a6ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f83a80beb50 0x7f83a8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7f5377277640 1 Processor -- start 2024-04-16T15:40:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7ffae759e640 1 Processor -- start 2024-04-16T15:40:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7ffae759e640 1 -- start start 2024-04-16T15:40:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83a6ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f83a80beb50 0x7f83a8155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51066/0 (socket says 172.21.15.71:51066) 2024-04-16T15:40:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83a6ffd640 1 -- 172.21.15.71:0/521657194 learned_addr learned my addr 172.21.15.71:0/521657194 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83a77fe640 1 --2- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a81531c0 0x7f83a80c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83a6ffd640 1 -- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a81531c0 msgr2=0x7f83a80c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83a6ffd640 1 --2- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a81531c0 0x7f83a80c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83a6ffd640 1 -- 172.21.15.71:0/521657194 >> v1:172.21.15.71:6789/0 conn(0x7f83a80c93c0 legacy=0x7f83a8156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f83a6ffd640 1 -- 172.21.15.71:0/521657194 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a8159be0 con 0x7f83a80beb50 2024-04-16T15:40:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f211eda8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f21181522b0 0x7f2118152720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7fbb01c7e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc0ac580 0x7fbafc14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7fbb01c7e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbafc0c1d80 0x7fbafc1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7fbb01c7e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbafc151df0 con 0x7fbafc1527f0 2024-04-16T15:40:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7fbb01c7e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fbafc14c320 con 0x7fbafc0c1d80 2024-04-16T15:40:43.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7fbb01c7e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbafc073620 con 0x7fbafc0ac580 2024-04-16T15:40:43.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7fbafbfff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fbafc1527f0 0x7fbafc06b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:60720/0 (socket says 172.21.15.71:60720) 2024-04-16T15:40:43.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7fbafbfff640 1 -- 172.21.15.71:0/3220558099 learned_addr learned my addr 172.21.15.71:0/3220558099 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:43.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7fc84b352640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fc84406fd50 0x7fc8440c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.288+0000 7f5377277640 1 -- start start 2024-04-16T15:40:43.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211eda8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2118151df0 con 0x7f21180ac580 2024-04-16T15:40:43.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211eda8640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f2118073620 con 0x7f21181522b0 2024-04-16T15:40:43.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211eda8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f211806e800 con 0x7f211814be70 2024-04-16T15:40:43.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7f5377277640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5370157300 0x7f5370073940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211cb44640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f211814be70 0x7f21180c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211cb44640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f211814be70 0x7f21180c1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34104/0 (socket says 172.21.15.71:34104) 2024-04-16T15:40:43.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211cb44640 1 -- 172.21.15.71:0/3109816119 learned_addr learned my addr 172.21.15.71:0/3109816119 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fc84b352640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8440c22c0 con 0x7fc8440ac620 2024-04-16T15:40:43.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fc84b352640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8440aa890 con 0x7fc84406e800 2024-04-16T15:40:43.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fc8498ef640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc84406e800 0x7fc844152440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:60740/0 (socket says 172.21.15.71:60740) 2024-04-16T15:40:43.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f5377277640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5370159ec0 0x7f537015a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fc8498ef640 1 -- 172.21.15.71:0/4491675 learned_addr learned my addr 172.21.15.71:0/4491675 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:43.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fc84b352640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fc844151fb0 con 0x7fc84406fd50 2024-04-16T15:40:43.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fc8490ee640 1 --2- 172.21.15.71:0/4491675 >> v2:172.21.15.71:3300/0 conn(0x7fc84406fd50 0x7fc8440c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fbafaffd640 1 --2- 172.21.15.71:0/3220558099 >> v2:172.21.15.122:3300/0 conn(0x7fbafc0c1d80 0x7fbafc1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fbafb7fe640 1 --2- 172.21.15.71:0/3220558099 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc0ac580 0x7fbafc14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211cb44640 1 -- 172.21.15.71:0/3109816119 >> v2:172.21.15.71:3300/0 conn(0x7f21181522b0 msgr2=0x7f2118152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211cb44640 1 --2- 172.21.15.71:0/3109816119 >> v2:172.21.15.71:3300/0 conn(0x7f21181522b0 0x7f2118152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f211cb44640 1 -- 172.21.15.71:0/3109816119 >> v1:172.21.15.71:6789/0 conn(0x7f21180ac580 legacy=0x7f211806bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459859561 0 0) 0x7fc8440aa890 con 0x7fc84406e800 2024-04-16T15:40:43.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc830003520 con 0x7fc84406e800 2024-04-16T15:40:43.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f83a77fe640 1 --2- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a81531c0 0x7f83a80c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f5377277640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53700c3b50 con 0x7f53700c34c0 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f5377277640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f53700721d0 con 0x7f5370159ec0 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f5377277640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f53700718a0 con 0x7f5370157300 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f5376275640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5370157300 0x7f5370073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7ffae759e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7ffae80c6e40 0x7ffae8161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7ffae759e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffae8159ce0 con 0x7ffae806bd80 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7ffae759e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffae80c7800 con 0x7ffae80b0970 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.289+0000 7ffae759e640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7ffae81599d0 con 0x7ffae80c6e40 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7ffae659c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ffae80b0970 0x7ffae81560b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:60726/0 (socket says 172.21.15.71:60726) 2024-04-16T15:40:43.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7ffae659c640 1 -- 172.21.15.71:0/2238662326 learned_addr learned my addr 172.21.15.71:0/2238662326 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:43.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7ffae6d9d640 1 --2- 172.21.15.71:0/2238662326 >> v2:172.21.15.71:3300/0 conn(0x7ffae80c6e40 0x7ffae8161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7ffae5d9b640 1 -- 172.21.15.71:0/2238662326 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209438955 0 0) 0x7ffae80c7800 con 0x7ffae80b0970 2024-04-16T15:40:43.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7ffae5d9b640 1 -- 172.21.15.71:0/2238662326 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffac8003520 con 0x7ffae80b0970 2024-04-16T15:40:43.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7f211cb44640 1 -- 172.21.15.71:0/3109816119 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21180738c0 con 0x7f211814be70 2024-04-16T15:40:43.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f5376275640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5370157300 0x7f5370073940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51082/0 (socket says 172.21.15.71:51082) 2024-04-16T15:40:43.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f5376275640 1 -- 172.21.15.71:0/78316095 learned_addr learned my addr 172.21.15.71:0/78316095 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.290+0000 7f5376a76640 1 --2- 172.21.15.71:0/78316095 >> v2:172.21.15.71:3300/0 conn(0x7f5370159ec0 0x7f537015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fbafaffd640 1 -- 172.21.15.71:0/3220558099 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc0ac580 msgr2=0x7fbafc14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7ffae6d9d640 1 -- 172.21.15.71:0/2238662326 >> v1:172.21.15.122:6789/0 conn(0x7ffae80b0970 legacy=0x7ffae81560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7ffae6d9d640 1 -- 172.21.15.71:0/2238662326 >> v1:172.21.15.71:6789/0 conn(0x7ffae806bd80 legacy=0x7ffae80c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7ffae6d9d640 1 -- 172.21.15.71:0/2238662326 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae80c7ae0 con 0x7ffae80c6e40 2024-04-16T15:40:43.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fec87d50640 1 Processor -- start 2024-04-16T15:40:43.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fbafaffd640 1 --2- 172.21.15.71:0/3220558099 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc0ac580 0x7fbafc14bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7f83a6ffd640 1 --2- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a80beb50 0x7f83a8155b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009e50 tx=0x7f83a002bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8025c856609d907 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7f5e98d59640 1 Processor -- start 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 214919295 0 0) 0x7fc830003520 con 0x7fc84406e800 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 >> v2:172.21.15.71:3300/0 conn(0x7fc84406fd50 msgr2=0x7fc8440c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 --2- 172.21.15.71:0/4491675 >> v2:172.21.15.71:3300/0 conn(0x7fc84406fd50 0x7fc8440c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 >> v1:172.21.15.71:6789/0 conn(0x7fc8440ac620 legacy=0x7fc84406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8440aab70 con 0x7fc84406e800 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fc83c0019f0 con 0x7fc84406e800 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 >> v1:172.21.15.122:6789/0 conn(0x7fc84406e800 legacy=0x7fc844152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc830003a00 0x7fc830003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc830004800 0x7fc830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc830005210 0x7fc83000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc844151fb0 con 0x7fc830003a00 2024-04-16T15:40:43.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc83000c150 con 0x7fc830005210 2024-04-16T15:40:43.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc83000c350 con 0x7fc830004800 2024-04-16T15:40:43.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fbafaffd640 1 -- 172.21.15.71:0/3220558099 >> v1:172.21.15.122:6789/0 conn(0x7fbafc1527f0 legacy=0x7fbafc06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7f5376275640 1 -- 172.21.15.71:0/78316095 >> v2:172.21.15.71:3300/0 conn(0x7f5370159ec0 msgr2=0x7f537015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7f5376275640 1 --2- 172.21.15.71:0/78316095 >> v2:172.21.15.71:3300/0 conn(0x7f5370159ec0 0x7f537015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7f5376275640 1 -- 172.21.15.71:0/78316095 >> v1:172.21.15.122:6789/0 conn(0x7f53700c34c0 legacy=0x7f537006ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7f5376275640 1 -- 172.21.15.71:0/78316095 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5370071b80 con 0x7f5370157300 2024-04-16T15:40:43.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7f5e98d59640 1 -- start start 2024-04-16T15:40:43.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fec87d50640 1 -- start start 2024-04-16T15:40:43.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f211cb44640 1 --2- 172.21.15.71:0/3109816119 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f211814be70 0x7f21180c1d80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f210c007360 tx=0x7f210c02c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e402b0894920de9e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc8488ed640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc830004800 0x7fc830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc8488ed640 1 -- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc830005210 msgr2=0x7fc83000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc8488ed640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc830005210 0x7fc83000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc8488ed640 1 -- 172.21.15.71:0/4491675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc830003a00 msgr2=0x7fc830003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc8488ed640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc830003a00 0x7fc830003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc8488ed640 1 -- 172.21.15.71:0/4491675 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc830003520 con 0x7fc830004800 2024-04-16T15:40:43.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fc8498ef640 1 -- 172.21.15.71:0/4491675 reap_dead start 2024-04-16T15:40:43.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f83a67fc640 1 -- 172.21.15.71:0/521657194 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a002c960 con 0x7f83a80beb50 2024-04-16T15:40:43.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f83a67fc640 1 -- 172.21.15.71:0/521657194 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a002cb00 con 0x7f83a80beb50 2024-04-16T15:40:43.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f83a67fc640 1 -- 172.21.15.71:0/521657194 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00343f0 con 0x7f83a80beb50 2024-04-16T15:40:43.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7ffae6d9d640 1 --2- 172.21.15.71:0/2238662326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae80c6e40 0x7ffae8161330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffae0014480 tx=0x7ffae0034450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49944ef2298071e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.291+0000 7fbafaffd640 1 -- 172.21.15.71:0/3220558099 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc06e800 con 0x7fbafc0c1d80 2024-04-16T15:40:43.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f5376275640 1 --2- 172.21.15.71:0/78316095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5370157300 0x7f5370073940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f537006f8b0 tx=0x7f536c02b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff88fda6f78d44b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7fec87d50640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fec780dbfe0 0x7fec7803ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f5e98d59640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5e9414c9c0 0x7f5e940c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f5e98d59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e94156420 0x7f5e940aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f5e98d59640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e94073f00 con 0x7f5e940ae220 2024-04-16T15:40:43.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f21177fe640 1 -- 172.21.15.71:0/3109816119 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c035020 con 0x7f211814be70 2024-04-16T15:40:43.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f5e98d59640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f5e94073a70 con 0x7f5e9414c9c0 2024-04-16T15:40:43.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f5e98d59640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5e94070a00 con 0x7f5e94156420 2024-04-16T15:40:43.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f21177fe640 1 -- 172.21.15.71:0/3109816119 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f210c008df0 con 0x7f211814be70 2024-04-16T15:40:43.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7f21177fe640 1 -- 172.21.15.71:0/3109816119 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c035020 con 0x7f211814be70 2024-04-16T15:40:43.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7ffae5d9b640 1 -- 172.21.15.71:0/2238662326 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0040070 con 0x7ffae80c6e40 2024-04-16T15:40:43.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7ffae5d9b640 1 -- 172.21.15.71:0/2238662326 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae0039710 con 0x7ffae80c6e40 2024-04-16T15:40:43.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7ffae5d9b640 1 -- 172.21.15.71:0/2238662326 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0039ae0 con 0x7ffae80c6e40 2024-04-16T15:40:43.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7fec87d50640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec78047560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7fec87d50640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fec7803f120 0x7fec78047aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f5375273640 1 -- 172.21.15.71:0/78316095 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c034070 con 0x7f5370157300 2024-04-16T15:40:43.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7fec87d50640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fec78006f80 con 0x7fec780dbfe0 2024-04-16T15:40:43.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7fec87d50640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fec78007520 con 0x7fec7803f120 2024-04-16T15:40:43.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.292+0000 7fec87d50640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fec7800baf0 con 0x7fec78004af0 2024-04-16T15:40:43.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fbafaffd640 1 --2- 172.21.15.71:0/3220558099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbafc0c1d80 0x7fbafc1522b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0097a0 tx=0x7fbaec02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7da6c0b626d2e163 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec852eb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec78047560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec852eb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec78047560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34132/0 (socket says 172.21.15.71:34132) 2024-04-16T15:40:43.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f5375273640 1 -- 172.21.15.71:0/78316095 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536c030870 con 0x7f5370157300 2024-04-16T15:40:43.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f5375273640 1 -- 172.21.15.71:0/78316095 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c030c40 con 0x7f5370157300 2024-04-16T15:40:43.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec85aec640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fec780dbfe0 0x7fec7803ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec85aec640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fec780dbfe0 0x7fec7803ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46864/0 (socket says 172.21.15.71:46864) 2024-04-16T15:40:43.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec85aec640 1 -- 172.21.15.71:0/857099057 learned_addr learned my addr 172.21.15.71:0/857099057 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f5e9259c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5e9414c9c0 0x7f5e940c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec862ed640 1 --2- 172.21.15.71:0/857099057 >> v2:172.21.15.122:3300/0 conn(0x7fec7803f120 0x7fec78047aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fbafa7fc640 1 -- 172.21.15.71:0/3220558099 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec031780 con 0x7fbafc0c1d80 2024-04-16T15:40:43.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fbafa7fc640 1 -- 172.21.15.71:0/3220558099 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaec037070 con 0x7fbafc0c1d80 2024-04-16T15:40:43.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fbafa7fc640 1 -- 172.21.15.71:0/3220558099 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec031b40 con 0x7fbafc0c1d80 2024-04-16T15:40:43.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f83adf59640 1 -- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a80beb50 msgr2=0x7f83a8155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f5e9259c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5e9414c9c0 0x7f5e940c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46858/0 (socket says 172.21.15.71:46858) 2024-04-16T15:40:43.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f5e9259c640 1 -- 172.21.15.71:0/2763101423 learned_addr learned my addr 172.21.15.71:0/2763101423 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f83adf59640 1 --2- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a80beb50 0x7f83a8155b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009e50 tx=0x7f83a002bdc0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f83adf59640 1 -- 172.21.15.71:0/521657194 shutdown_connections 2024-04-16T15:40:43.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f83adf59640 1 --2- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a80beb50 0x7f83a8155b40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f83adf59640 1 --2- 172.21.15.71:0/521657194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a81531c0 0x7f83a80c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f83adf59640 1 -- 172.21.15.71:0/521657194 >> 172.21.15.71:0/521657194 conn(0x7f83a806bb10 msgr2=0x7f83a806bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f83adf59640 1 -- 172.21.15.71:0/521657194 shutdown_connections 2024-04-16T15:40:43.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec85aec640 1 -- 172.21.15.71:0/857099057 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 msgr2=0x7fec78047560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec85aec640 1 --2- 172.21.15.71:0/857099057 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec78047560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec85aec640 1 -- 172.21.15.71:0/857099057 >> v2:172.21.15.122:3300/0 conn(0x7fec7803f120 msgr2=0x7fec78047aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7fec85aec640 1 --2- 172.21.15.71:0/857099057 >> v2:172.21.15.122:3300/0 conn(0x7fec7803f120 0x7fec78047aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7fec85aec640 1 -- 172.21.15.71:0/857099057 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7800bdd0 con 0x7fec780dbfe0 2024-04-16T15:40:43.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5377277640 1 -- 172.21.15.71:0/78316095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5370157300 msgr2=0x7f5370073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5377277640 1 --2- 172.21.15.71:0/78316095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5370157300 0x7f5370073940 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f537006f8b0 tx=0x7f536c02b9b0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7ffae759e640 1 -- 172.21.15.71:0/2238662326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae80c6e40 msgr2=0x7ffae8161330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7ffae759e640 1 --2- 172.21.15.71:0/2238662326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae80c6e40 0x7ffae8161330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffae0014480 tx=0x7ffae0034450 comp rx=0 tx=0).stop 2024-04-16T15:40:43.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7ffae759e640 1 -- 172.21.15.71:0/2238662326 shutdown_connections 2024-04-16T15:40:43.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7ffae759e640 1 --2- 172.21.15.71:0/2238662326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae80c6e40 0x7ffae8161330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7ffae759e640 1 -- 172.21.15.71:0/2238662326 >> 172.21.15.71:0/2238662326 conn(0x7ffae80c7f50 msgr2=0x7ffae80c8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7ffae759e640 1 -- 172.21.15.71:0/2238662326 shutdown_connections 2024-04-16T15:40:43.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f211eda8640 1 -- 172.21.15.71:0/3109816119 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f211814be70 msgr2=0x7f21180c1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f211eda8640 1 --2- 172.21.15.71:0/3109816119 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f211814be70 0x7f21180c1d80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f210c007360 tx=0x7f210c02c980 comp rx=0 tx=0).stop 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f211eda8640 1 -- 172.21.15.71:0/3109816119 shutdown_connections 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f211eda8640 1 --2- 172.21.15.71:0/3109816119 >> v2:172.21.15.71:3300/0 conn(0x7f21181522b0 0x7f2118152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f211eda8640 1 --2- 172.21.15.71:0/3109816119 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f211814be70 0x7f21180c1d80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f211eda8640 1 -- 172.21.15.71:0/3109816119 >> 172.21.15.71:0/3109816119 conn(0x7f21180c7560 msgr2=0x7f21180c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5377277640 1 -- 172.21.15.71:0/78316095 shutdown_connections 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5377277640 1 --2- 172.21.15.71:0/78316095 >> v2:172.21.15.71:3300/0 conn(0x7f5370159ec0 0x7f537015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5377277640 1 --2- 172.21.15.71:0/78316095 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5370157300 0x7f5370073940 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5377277640 1 -- 172.21.15.71:0/78316095 >> 172.21.15.71:0/78316095 conn(0x7f53700c7480 msgr2=0x7f53700c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7fec862ed640 1 --2- 172.21.15.71:0/857099057 >> v2:172.21.15.122:3300/0 conn(0x7fec7803f120 0x7fec78047aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:43.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7fec852eb640 1 --2- 172.21.15.71:0/857099057 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec78047560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:43.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5377277640 1 -- 172.21.15.71:0/78316095 shutdown_connections 2024-04-16T15:40:43.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7fc8488ed640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc830004800 0x7fc830004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc840007970 tx=0x7fc830005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28d72d38ea57eaab server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7ffae759e640 1 -- 172.21.15.71:0/2238662326 wait complete. 2024-04-16T15:40:43.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.293+0000 7f83adf59640 1 -- 172.21.15.71:0/521657194 wait complete. 2024-04-16T15:40:43.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840041070 con 0x7fc830004800 2024-04-16T15:40:43.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc840034950 con 0x7fc830004800 2024-04-16T15:40:43.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7fc83bfff640 1 -- 172.21.15.71:0/4491675 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840003980 con 0x7fc830004800 2024-04-16T15:40:43.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5e9259c640 1 -- 172.21.15.71:0/2763101423 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e94156420 msgr2=0x7f5e940aa5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T15:40:43.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5e9259c640 1 --2- 172.21.15.71:0/2763101423 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e94156420 0x7f5e940aa5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5e9259c640 1 -- 172.21.15.71:0/2763101423 >> v1:172.21.15.71:6789/0 conn(0x7f5e940ae220 legacy=0x7f5e94155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5e9259c640 1 -- 172.21.15.71:0/2763101423 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e940c2d20 con 0x7f5e9414c9c0 2024-04-16T15:40:43.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3220558099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbafc0c1d80 msgr2=0x7fbafc1522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fbb01c7e640 1 --2- 172.21.15.71:0/3220558099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbafc0c1d80 0x7fbafc1522b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0097a0 tx=0x7fbaec02b820 comp rx=0 tx=0).stop 2024-04-16T15:40:43.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.294+0000 7f5377277640 1 -- 172.21.15.71:0/78316095 wait complete. 2024-04-16T15:40:43.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7f211eda8640 1 -- 172.21.15.71:0/3109816119 shutdown_connections 2024-04-16T15:40:43.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3220558099 shutdown_connections 2024-04-16T15:40:43.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fbb01c7e640 1 --2- 172.21.15.71:0/3220558099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbafc0c1d80 0x7fbafc1522b0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0097a0 tx=0x7fbaec02b820 comp rx=0 tx=0).stop 2024-04-16T15:40:43.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fbb01c7e640 1 --2- 172.21.15.71:0/3220558099 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc0ac580 0x7fbafc14bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3220558099 >> 172.21.15.71:0/3220558099 conn(0x7fbafc0c7560 msgr2=0x7fbafc0c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7f5e9259c640 1 --2- 172.21.15.71:0/2763101423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 0x7f5e940c47d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00b1a0 tx=0x7f5e8c02c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e5cb31196a87fd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fec85aec640 1 --2- 172.21.15.71:0/857099057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec780dbfe0 0x7fec7803ebe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec780e2e90 tx=0x7fec7002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0e3e41f1fb7a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7f83adf59640 1 Processor -- start 2024-04-16T15:40:43.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fc84b352640 1 -- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc830004800 msgr2=0x7fc830004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fc84b352640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc830004800 0x7fc830004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc840007970 tx=0x7fc830005830 comp rx=0 tx=0).stop 2024-04-16T15:40:43.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fec84aea640 1 -- 172.21.15.71:0/857099057 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70035020 con 0x7fec780dbfe0 2024-04-16T15:40:43.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3220558099 shutdown_connections 2024-04-16T15:40:43.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fec84aea640 1 -- 172.21.15.71:0/857099057 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec70031880 con 0x7fec780dbfe0 2024-04-16T15:40:43.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fec84aea640 1 -- 172.21.15.71:0/857099057 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70031c50 con 0x7fec780dbfe0 2024-04-16T15:40:43.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3220558099 wait complete. 2024-04-16T15:40:43.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7f5e9159a640 1 -- 172.21.15.71:0/2763101423 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c02cae0 con 0x7f5e9414c9c0 2024-04-16T15:40:43.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7f5e9159a640 1 -- 172.21.15.71:0/2763101423 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e8c02cc80 con 0x7f5e9414c9c0 2024-04-16T15:40:43.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7f5e9159a640 1 -- 172.21.15.71:0/2763101423 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c02cae0 con 0x7f5e9414c9c0 2024-04-16T15:40:43.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7f83adf59640 1 -- start start 2024-04-16T15:40:43.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.295+0000 7f211eda8640 1 -- 172.21.15.71:0/3109816119 wait complete. 2024-04-16T15:40:43.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fc84b352640 1 -- 172.21.15.71:0/4491675 shutdown_connections 2024-04-16T15:40:43.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fc84b352640 1 --2- 172.21.15.71:0/4491675 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc830004800 0x7fc830004c80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fc84b352640 1 -- 172.21.15.71:0/4491675 >> 172.21.15.71:0/4491675 conn(0x7fc8440a9e20 msgr2=0x7fc8440c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7ffae759e640 1 Processor -- start 2024-04-16T15:40:43.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7f5377277640 1 Processor -- start 2024-04-16T15:40:43.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fc84b352640 1 -- 172.21.15.71:0/4491675 shutdown_connections 2024-04-16T15:40:43.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fc84b352640 1 -- 172.21.15.71:0/4491675 wait complete. 2024-04-16T15:40:43.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7f5377277640 1 -- start start 2024-04-16T15:40:43.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7ffae759e640 1 -- start start 2024-04-16T15:40:43.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fec87d50640 1 -- 172.21.15.71:0/857099057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec780dbfe0 msgr2=0x7fec7803ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fec87d50640 1 --2- 172.21.15.71:0/857099057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec780dbfe0 0x7fec7803ebe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec780e2e90 tx=0x7fec7002b920 comp rx=0 tx=0).stop 2024-04-16T15:40:43.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fec87d50640 1 -- 172.21.15.71:0/857099057 shutdown_connections 2024-04-16T15:40:43.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fec87d50640 1 --2- 172.21.15.71:0/857099057 >> v2:172.21.15.122:3300/0 conn(0x7fec7803f120 0x7fec78047aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fec87d50640 1 --2- 172.21.15.71:0/857099057 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec78047560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.296+0000 7fec87d50640 1 --2- 172.21.15.71:0/857099057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec780dbfe0 0x7fec7803ebe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7fec87d50640 1 -- 172.21.15.71:0/857099057 >> 172.21.15.71:0/857099057 conn(0x7fec780062d0 msgr2=0x7fec780066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83adf59640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a80beb50 0x7f83a80c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f5e98d59640 1 -- 172.21.15.71:0/2763101423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 msgr2=0x7f5e940c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f5e98d59640 1 --2- 172.21.15.71:0/2763101423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 0x7f5e940c47d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00b1a0 tx=0x7f5e8c02c140 comp rx=0 tx=0).stop 2024-04-16T15:40:43.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7fec87d50640 1 -- 172.21.15.71:0/857099057 shutdown_connections 2024-04-16T15:40:43.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83adf59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a80c93c0 0x7f83a80c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae806bd80 0x7ffae814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae80b0970 0x7ffae80aac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83adf59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a81531c0 0x7f83a81f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae759e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffae80c6e40 0x7ffae814c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae759e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffae80c9660 con 0x7ffae806bd80 2024-04-16T15:40:43.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae759e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffae80c94e0 con 0x7ffae80b0970 2024-04-16T15:40:43.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae759e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ffae81567c0 con 0x7ffae80c6e40 2024-04-16T15:40:43.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83adf59640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83a80cbce0 con 0x7f83a80beb50 2024-04-16T15:40:43.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83adf59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83a80cba00 con 0x7f83a81531c0 2024-04-16T15:40:43.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83adf59640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f83a806e650 con 0x7f83a80c93c0 2024-04-16T15:40:43.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffaddd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae80b0970 0x7ffae80aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a81531c0 0x7f83a81f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a81531c0 0x7f83a81f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51096/0 (socket says 172.21.15.71:51096) 2024-04-16T15:40:43.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83a7fff640 1 -- 172.21.15.71:0/124617506 learned_addr learned my addr 172.21.15.71:0/124617506 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae6d9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffae80c6e40 0x7ffae814c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae6d9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffae80c6e40 0x7ffae814c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34158/0 (socket says 172.21.15.71:34158) 2024-04-16T15:40:43.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7ffae6d9d640 1 -- 172.21.15.71:0/3841918430 learned_addr learned my addr 172.21.15.71:0/3841918430 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f83a6ffd640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a80c93c0 0x7f83a80c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7fbb01c7e640 1 Processor -- start 2024-04-16T15:40:43.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7fec87d50640 1 -- 172.21.15.71:0/857099057 wait complete. 2024-04-16T15:40:43.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7f5377277640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53700c34c0 0x7f53701f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.297+0000 7fc84b352640 1 Processor -- start 2024-04-16T15:40:43.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5377277640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5370157300 0x7f53701f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7fc84b352640 1 -- start start 2024-04-16T15:40:43.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f83a6ffd640 1 -- 172.21.15.71:0/124617506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a81531c0 msgr2=0x7f83a81f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5376275640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53700c34c0 0x7f53701f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5376275640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53700c34c0 0x7f53701f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51112/0 (socket says 172.21.15.71:51112) 2024-04-16T15:40:43.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae6d9d640 1 -- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae80b0970 msgr2=0x7ffae80aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7fbb01c7e640 1 -- start start 2024-04-16T15:40:43.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae6d9d640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae80b0970 0x7ffae80aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae6d9d640 1 -- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae806bd80 msgr2=0x7ffae814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae659c640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae806bd80 0x7ffae814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f83a6ffd640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83a81531c0 0x7f83a81f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5e98d59640 1 -- 172.21.15.71:0/2763101423 shutdown_connections 2024-04-16T15:40:43.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5e98d59640 1 --2- 172.21.15.71:0/2763101423 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e94156420 0x7f5e940aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5e98d59640 1 --2- 172.21.15.71:0/2763101423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 0x7f5e940c47d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae6d9d640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae806bd80 0x7ffae814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae6d9d640 1 -- 172.21.15.71:0/3841918430 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae80ab6d0 con 0x7ffae80c6e40 2024-04-16T15:40:43.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5e98d59640 1 -- 172.21.15.71:0/2763101423 >> 172.21.15.71:0/2763101423 conn(0x7f5e9406f800 msgr2=0x7f5e9406fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae659c640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffae806bd80 0x7ffae814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:43.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5375a74640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5370157300 0x7f53701f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5375a74640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5370157300 0x7f53701f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34178/0 (socket says 172.21.15.71:34178) 2024-04-16T15:40:43.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5377277640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5370159ec0 0x7f53700c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5376275640 1 -- 172.21.15.71:0/944734986 learned_addr learned my addr 172.21.15.71:0/944734986 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f83a77fe640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a80beb50 0x7f83a80c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f83a6ffd640 1 -- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a80beb50 msgr2=0x7f83a80c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae6d9d640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffae80c6e40 0x7ffae814c680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffae0014480 tx=0x7ffae0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5376a76640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5370159ec0 0x7f53700c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f83a6ffd640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a80beb50 0x7f83a80c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f83a6ffd640 1 -- 172.21.15.71:0/124617506 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a80aca20 con 0x7f83a80c93c0 2024-04-16T15:40:43.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f83a77fe640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a80beb50 0x7f83a80c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:43.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae004a020 con 0x7ffae80c6e40 2024-04-16T15:40:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f537006f190 con 0x7f5370159ec0 2024-04-16T15:40:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53700c2960 con 0x7f53700c34c0 2024-04-16T15:40:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f537006f310 con 0x7f5370157300 2024-04-16T15:40:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae0045870 con 0x7ffae80c6e40 2024-04-16T15:40:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0040040 con 0x7ffae80c6e40 2024-04-16T15:40:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.298+0000 7ffae759e640 1 -- 172.21.15.71:0/3841918430 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae80ab990 con 0x7ffae80c6e40 2024-04-16T15:40:43.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f211eda8640 1 Processor -- start 2024-04-16T15:40:43.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fec87d50640 1 Processor -- start 2024-04-16T15:40:43.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc84b352640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc830004800 0x7fc8441532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc84b352640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8441ebae0 0x7fc844155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7ffae759e640 1 -- 172.21.15.71:0/3841918430 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae8157d60 con 0x7ffae80c6e40 2024-04-16T15:40:43.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f5e98d59640 1 -- 172.21.15.71:0/2763101423 shutdown_connections 2024-04-16T15:40:43.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f5e98d59640 1 -- 172.21.15.71:0/2763101423 wait complete. 2024-04-16T15:40:43.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f211eda8640 1 -- start start 2024-04-16T15:40:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbb01c7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbafc0ac580 0x7fbafc0c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbb01c7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbafc1527f0 0x7fbafc072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbb01c7e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc06f550 0x7fbafc06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbb01c7e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc073e80 con 0x7fbafc1527f0 2024-04-16T15:40:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbb01c7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc073ba0 con 0x7fbafc0ac580 2024-04-16T15:40:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbb01c7e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbafc0c2200 con 0x7fbafc06f550 2024-04-16T15:40:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc84b352640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8441e8430 0x7fc84414a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbafbfff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc06f550 0x7fbafc06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbafbfff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc06f550 0x7fbafc06ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34194/0 (socket says 172.21.15.71:34194) 2024-04-16T15:40:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbafbfff640 1 -- 172.21.15.71:0/3765572767 learned_addr learned my addr 172.21.15.71:0/3765572767 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fbafaffd640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbafc1527f0 0x7fbafc072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc84b352640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc84406bab0 con 0x7fc8441ebae0 2024-04-16T15:40:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc84b352640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc84406b930 con 0x7fc830004800 2024-04-16T15:40:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc84b352640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc8440c1d80 con 0x7fc8441e8430 2024-04-16T15:40:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc8488ed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8441ebae0 0x7fc844155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc8488ed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8441ebae0 0x7fc844155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46902/0 (socket says 172.21.15.71:46902) 2024-04-16T15:40:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fec87d50640 1 -- start start 2024-04-16T15:40:43.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f5376275640 1 -- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5370157300 msgr2=0x7f53701f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f5376275640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5370157300 0x7f53701f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7ffae759e640 1 -- 172.21.15.71:0/3841918430 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffae80744e0 con 0x7ffae80c6e40 2024-04-16T15:40:43.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fec87d50640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec780d5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fec87d50640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7803f120 0x7fec780d67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fec87d50640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec780dbfe0 0x7fec78175030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fec87d50640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800b8b0 con 0x7fec7803f120 2024-04-16T15:40:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fec87d50640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800b730 con 0x7fec780dbfe0 2024-04-16T15:40:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fec87d50640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fec7802e4c0 con 0x7fec78004af0 2024-04-16T15:40:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f5376275640 1 -- 172.21.15.71:0/944734986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5370159ec0 msgr2=0x7f53700c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f5376275640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5370159ec0 0x7f53700c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7f5376275640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53700aafb0 con 0x7f53700c34c0 2024-04-16T15:40:43.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7f83a6ffd640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a80c93c0 0x7f83a80c5100 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f83a80cc440 tx=0x7f83a0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7f211eda8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21180ac580 0x7f211814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7f211eda8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f211814be70 0x7f211814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7f211eda8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21181522b0 0x7f2118144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fbafbfff640 1 -- 172.21.15.71:0/3765572767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbafc0ac580 msgr2=0x7fbafc0c6b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T15:40:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fbafbfff640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbafc0ac580 0x7fbafc0c6b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fbafbfff640 1 -- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbafc1527f0 msgr2=0x7fbafc072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fbafbfff640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbafc1527f0 0x7fbafc072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fbafbfff640 1 -- 172.21.15.71:0/3765572767 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc146f70 con 0x7fbafc06f550 2024-04-16T15:40:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fbafaffd640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbafc1527f0 0x7fbafc072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7f211eda8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21180c22c0 con 0x7f21181522b0 2024-04-16T15:40:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0043020 con 0x7f83a80c93c0 2024-04-16T15:40:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a003e450 con 0x7f83a80c93c0 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc8488ed640 1 -- 172.21.15.71:0/1325571056 learned_addr learned my addr 172.21.15.71:0/1325571056 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc8490ee640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc830004800 0x7fc8441532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.299+0000 7fc8498ef640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8441e8430 0x7fc84414a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fc8488ed640 1 -- 172.21.15.71:0/1325571056 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8441e8430 msgr2=0x7fc84414a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fc8488ed640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8441e8430 0x7fc84414a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fc8488ed640 1 -- 172.21.15.71:0/1325571056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc830004800 msgr2=0x7fc8441532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fc8488ed640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc830004800 0x7fc8441532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fc8488ed640 1 -- 172.21.15.71:0/1325571056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8441e7c70 con 0x7fc8441ebae0 2024-04-16T15:40:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fc8488ed640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8441ebae0 0x7fc844155390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc8400343c0 tx=0x7fc8400036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f211cb44640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21180ac580 0x7f211814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f211eda8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2118073e80 con 0x7f211814be70 2024-04-16T15:40:43.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f2117fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f211814be70 0x7f211814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fbafbfff640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc06f550 0x7fbafc06ec90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0015be0 tx=0x7fbaf0013560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f5376275640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53700c34c0 0x7f53701f2ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f536c000c00 tx=0x7f536c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.300+0000 7fec862ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec780dbfe0 0x7fec78175030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec862ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec780dbfe0 0x7fec78175030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51136/0 (socket says 172.21.15.71:51136) 2024-04-16T15:40:43.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec862ed640 1 -- 172.21.15.71:0/632809661 learned_addr learned my addr 172.21.15.71:0/632809661 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec85aec640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec780d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f211eda8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f211814c250 con 0x7f21180ac580 2024-04-16T15:40:43.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f211cb44640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21180ac580 0x7f211814d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34222/0 (socket says 172.21.15.71:34222) 2024-04-16T15:40:43.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f211cb44640 1 -- 172.21.15.71:0/542549399 learned_addr learned my addr 172.21.15.71:0/542549399 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec852eb640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7803f120 0x7fec780d67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f5e98d59640 1 Processor -- start 2024-04-16T15:40:43.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f211d345640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21181522b0 0x7f2118144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c048070 con 0x7f53700c34c0 2024-04-16T15:40:43.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec862ed640 1 -- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 msgr2=0x7fec780d5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec862ed640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec78004af0 0x7fec780d5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf001d020 con 0x7fbafc06f550 2024-04-16T15:40:43.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec862ed640 1 -- 172.21.15.71:0/632809661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7803f120 msgr2=0x7fec780d67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec862ed640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7803f120 0x7fec780d67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536c030670 con 0x7f53700c34c0 2024-04-16T15:40:43.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf0019430 con 0x7fbafc06f550 2024-04-16T15:40:43.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fec862ed640 1 -- 172.21.15.71:0/632809661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec78174b70 con 0x7fec780dbfe0 2024-04-16T15:40:43.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf001d020 con 0x7fbafc06f550 2024-04-16T15:40:43.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f5e98d59640 1 -- start start 2024-04-16T15:40:43.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc146b20 con 0x7fbafc06f550 2024-04-16T15:40:43.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a003e840 con 0x7f83a80c93c0 2024-04-16T15:40:43.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc14c700 con 0x7fbafc06f550 2024-04-16T15:40:43.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f83adf59640 1 -- 172.21.15.71:0/124617506 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a80ad290 con 0x7f83a80c93c0 2024-04-16T15:40:43.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f211d345640 1 -- 172.21.15.71:0/542549399 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21180ac580 msgr2=0x7f211814d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7f83adf59640 1 -- 172.21.15.71:0/124617506 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a80beff0 con 0x7f83a80c93c0 2024-04-16T15:40:43.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f211d345640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21180ac580 0x7f211814d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7fec862ed640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec780dbfe0 0x7fec78175030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec8006a920 tx=0x7fec80075650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f211d345640 1 -- 172.21.15.71:0/542549399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f211814be70 msgr2=0x7f211814da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f211d345640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f211814be70 0x7f211814da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f211d345640 1 -- 172.21.15.71:0/542549399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2118145f20 con 0x7f21181522b0 2024-04-16T15:40:43.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.301+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffae004d020 con 0x7ffae80c6e40 2024-04-16T15:40:43.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7ffadf7fe640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffacc06c9c0 0x7ffacc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ffae00b68a0 con 0x7ffae80c6e40 2024-04-16T15:40:43.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7ffadf7fe640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7ffacc0760c0 0x7ffacc078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 --> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffae0011eb0 con 0x7ffacc0760c0 2024-04-16T15:40:43.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffae0080510 con 0x7ffae80c6e40 2024-04-16T15:40:43.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c034040 con 0x7f53700c34c0 2024-04-16T15:40:43.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f2117fff640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f211814be70 0x7f211814da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:43.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e98d59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e940ae220 0x7f5e941f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e98d59640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 0x7f5e9406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7ffae659c640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffacc06c9c0 0x7ffacc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e98d59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e94156420 0x7f5e9414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e98d59640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9414cda0 con 0x7f5e9414c9c0 2024-04-16T15:40:43.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e98d59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e941568c0 con 0x7f5e94156420 2024-04-16T15:40:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e98d59640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5e94156ba0 con 0x7f5e940ae220 2024-04-16T15:40:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e940ae220 0x7f5e941f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e9259c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e940ae220 0x7f5e941f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34224/0 (socket says 172.21.15.71:34224) 2024-04-16T15:40:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e9259c640 1 -- 172.21.15.71:0/3638336677 learned_addr learned my addr 172.21.15.71:0/3638336677 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7f5e92d9d640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e94156420 0x7f5e9414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f5e91d9b640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 0x7f5e9406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.302+0000 7ffaddd9b640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7ffacc0760c0 0x7ffacc078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80099070 con 0x7fec780dbfe0 2024-04-16T15:40:43.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f211d345640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21181522b0 0x7f2118144270 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f211000be20 tx=0x7f211000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec800748f0 con 0x7fec780dbfe0 2024-04-16T15:40:43.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84004c030 con 0x7fc8441ebae0 2024-04-16T15:40:43.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc840034750 con 0x7fc8441ebae0 2024-04-16T15:40:43.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840041040 con 0x7fc8441ebae0 2024-04-16T15:40:43.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7fc84b352640 1 -- 172.21.15.71:0/1325571056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8440c0e60 con 0x7fc8441ebae0 2024-04-16T15:40:43.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f83adf59640 1 -- 172.21.15.71:0/124617506 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83a8068be0 con 0x7f83a80c93c0 2024-04-16T15:40:43.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f5e9259c640 1 -- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e94156420 msgr2=0x7f5e9414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f5e9259c640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e94156420 0x7f5e9414f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f5e9259c640 1 -- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 msgr2=0x7f5e9406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f5e9259c640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 0x7f5e9406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f5e9259c640 1 -- 172.21.15.71:0/3638336677 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e94150e00 con 0x7f5e940ae220 2024-04-16T15:40:43.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f5e91d9b640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e9414c9c0 0x7f5e9406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:43.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7fc84b352640 1 -- 172.21.15.71:0/1325571056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8441e7f10 con 0x7fc8441ebae0 2024-04-16T15:40:43.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110019070 con 0x7f21181522b0 2024-04-16T15:40:43.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2110014440 con 0x7f21181522b0 2024-04-16T15:40:43.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110014830 con 0x7f21181522b0 2024-04-16T15:40:43.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7ffaddd9b640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7ffacc0760c0 0x7ffacc078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec800943f0 con 0x7fec780dbfe0 2024-04-16T15:40:43.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec781785a0 con 0x7fec780dbfe0 2024-04-16T15:40:43.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec78178b60 con 0x7fec780dbfe0 2024-04-16T15:40:43.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7ffae659c640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffacc06c9c0 0x7ffacc06ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffad80011e0 tx=0x7ffad8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53701f3b00 con 0x7f53700c34c0 2024-04-16T15:40:43.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7f5e9259c640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e940ae220 0x7f5e941f3f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000f80 tx=0x7f5e8c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53701e37a0 con 0x7f53700c34c0 2024-04-16T15:40:43.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2118155500 con 0x7f21181522b0 2024-04-16T15:40:43.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.303+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2118146200 con 0x7f21181522b0 2024-04-16T15:40:43.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc820001030 con 0x7fc8441ebae0 2024-04-16T15:40:43.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.304+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbaf0020020 con 0x7fbafc06f550 2024-04-16T15:40:43.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7fbae3fff640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbae806cd20 0x7fbae806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 <== osd.6 v2:172.21.15.122:6808/1958778321 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffae0011eb0 con 0x7ffacc0760c0 2024-04-16T15:40:43.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbadc001030 con 0x7fbafc06f550 2024-04-16T15:40:43.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21180aabb0 con 0x7f21181522b0 2024-04-16T15:40:43.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f211001c070 con 0x7f21181522b0 2024-04-16T15:40:43.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f21157fa640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20fc06c9c0 0x7f20fc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f211cb44640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20fc06c9c0 0x7f20fc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f211008bd40 con 0x7f21181522b0 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f21157fa640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7f20fc076130 0x7f20fc078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 --> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20fc078c40 con 0x7f20fc076130 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2110090030 con 0x7f21181522b0 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbaf0013c70 con 0x7fbafc06f550 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fbae3fff640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fbae8076ff0 0x7fbae8079490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 --> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbae8079b60 con 0x7fbae8076ff0 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fbafb7fe640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbae806cd20 0x7fbae806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbaf005e570 con 0x7fbafc06f550 2024-04-16T15:40:43.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7fbafaffd640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fbae8076ff0 0x7fbae8079490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7fbafb7fe640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbae806cd20 0x7fbae806f1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400e1e0 tx=0x7fbaf400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7fbafaffd640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fbae8076ff0 0x7fbae8079490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83a0046020 con 0x7f83a80c93c0 2024-04-16T15:40:43.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f838ffff640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f839406c9c0 0x7f839406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f536c04b070 con 0x7f53700c34c0 2024-04-16T15:40:43.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f5366ffd640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f535406c970 0x7f535406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5370068a60 con 0x7f53700c34c0 2024-04-16T15:40:43.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f536c0adfb0 con 0x7f53700c34c0 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f5366ffd640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f5354076070 0x7f53540784b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5354078b80 con 0x7f5354076070 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f536c077e80 con 0x7f53700c34c0 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec78178f60 con 0x7fec780dbfe0 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec8009c360 con 0x7fec780dbfe0 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7fec767fc640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec6406c9c0 0x7fec6406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fec80106d00 con 0x7fec780dbfe0 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7fec85aec640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec6406c9c0 0x7fec6406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7fec767fc640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7fec64076130 0x7fec64078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 --> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec80098ec0 con 0x7fec64076130 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fec80109360 con 0x7fec780dbfe0 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7fec852eb640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7fec64076130 0x7fec64078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c043020 con 0x7f5e940ae220 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e8c03e660 con 0x7f5e940ae220 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c02cae0 con 0x7f5e940ae220 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e94144dd0 con 0x7f5e940ae220 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.305+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9414d490 con 0x7f5e940ae220 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e94153630 con 0x7f5e940ae220 2024-04-16T15:40:43.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f5375a74640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f535406c970 0x7f535406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 <== osd.2 v2:172.21.15.71:6813/3827429750 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbae8079b60 con 0x7fbae8076ff0 2024-04-16T15:40:43.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f83a00ae620 con 0x7f83a80c93c0 2024-04-16T15:40:43.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f838ffff640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f8394076130 0x7f8394078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 --> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8394078c40 con 0x7f8394076130 2024-04-16T15:40:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc84004f070 con 0x7fc8441ebae0 2024-04-16T15:40:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fc839ffb640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc82806c9c0 0x7fc82806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc8400b5fa0 con 0x7fc8441ebae0 2024-04-16T15:40:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fc839ffb640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fc828076140 0x7fc828078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 --> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc840048e50 con 0x7fc828076140 2024-04-16T15:40:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.306+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc8400bb590 con 0x7fc8441ebae0 2024-04-16T15:40:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.307+0000 7fc8490ee640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc82806c9c0 0x7fc82806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7fc8490ee640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc82806c9c0 0x7fc82806ee80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc834006fd0 tx=0x7fc834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7fc8498ef640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fc828076140 0x7fc828078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7f5376a76640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f5354076070 0x7f53540784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f2117fff640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7f20fc076130 0x7f20fc078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f211cb44640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20fc06c9c0 0x7f20fc06ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f210c02c9c0 tx=0x7f210c00a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7f2117fff640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7f20fc076130 0x7f20fc078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7f5375a74640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f535406c970 0x7f535406ee30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f537014cc20 tx=0x7f5368009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f83a00b5050 con 0x7f83a80c93c0 2024-04-16T15:40:43.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.308+0000 7f83a77fe640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f839406c9c0 0x7f839406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7f83a7fff640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f8394076130 0x7f8394078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7fec852eb640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7fec64076130 0x7fec64078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7fc8498ef640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fc828076140 0x7fc828078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7f83a7fff640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f8394076130 0x7f8394078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7f83a77fe640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f839406c9c0 0x7f839406ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8398009a80 tx=0x7f8398008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7fec85aec640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec6406c9c0 0x7fec6406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec70036000 tx=0x7fec7003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.309+0000 7f5376a76640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f5354076070 0x7f53540784b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.310+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 <== osd.0 v2:172.21.15.71:6800/2582424538 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20fc078c40 con 0x7f20fc076130 2024-04-16T15:40:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.310+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 <== osd.3 v2:172.21.15.71:6824/2052589418 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8394078c40 con 0x7f8394076130 2024-04-16T15:40:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.310+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 <== osd.1 v2:172.21.15.71:6808/357157073 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc840048e50 con 0x7fc828076140 2024-04-16T15:40:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.310+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 <== osd.5 v2:172.21.15.122:6817/1133463708 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec80098ec0 con 0x7fec64076130 2024-04-16T15:40:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.310+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 <== osd.4 v2:172.21.15.122:6816/710391503 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5354078b80 con 0x7f5354076070 2024-04-16T15:40:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.311+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e8c046020 con 0x7f5e940ae220 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.311+0000 7f5e82ffd640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5e7406c9f0 0x7f5e7406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.312+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5e8c0aeb50 con 0x7f5e940ae220 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.312+0000 7f5e82ffd640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f5e74076170 0x7f5e740785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.312+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 --> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e8c02cc80 con 0x7f5e74076170 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.312+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5e8c0aef40 con 0x7f5e940ae220 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.312+0000 7f5e91d9b640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5e7406c9f0 0x7f5e7406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.313+0000 7f5e92d9d640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f5e74076170 0x7f5e740785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.313+0000 7f5e91d9b640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5e7406c9f0 0x7f5e7406eeb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e94150320 tx=0x7f5e88007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.315+0000 7f5e92d9d640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f5e74076170 0x7f5e740785b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.315+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 <== osd.7 v2:172.21.15.122:6800/3651959178 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e8c02cc80 con 0x7f5e74076170 2024-04-16T15:40:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.336+0000 7ffae759e640 1 -- 172.21.15.71:0/3841918430 --> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffae80c5610 con 0x7ffacc0760c0 2024-04-16T15:40:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.337+0000 7ffadf7fe640 1 -- 172.21.15.71:0/3841918430 <== osd.6 v2:172.21.15.122:6808/1958778321 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffae80c5610 con 0x7ffacc0760c0 2024-04-16T15:40:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.338+0000 7ffadcd99640 1 -- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7ffacc0760c0 msgr2=0x7ffacc078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.338+0000 7ffadcd99640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7ffacc0760c0 0x7ffacc078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.338+0000 7ffadcd99640 1 -- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffacc06c9c0 msgr2=0x7ffacc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.338+0000 7ffadcd99640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffacc06c9c0 0x7ffacc06ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffad80011e0 tx=0x7ffad8009040 comp rx=0 tx=0).stop 2024-04-16T15:40:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.338+0000 7ffadcd99640 1 -- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffae80c6e40 msgr2=0x7ffae814c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.339+0000 7ffadcd99640 1 --2- 172.21.15.71:0/3841918430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffae80c6e40 0x7ffae814c680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffae0014480 tx=0x7ffae0004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.339+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 --> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc820002dd0 con 0x7fc828076140 2024-04-16T15:40:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.339+0000 7ffae6d9d640 1 -- 172.21.15.71:0/3841918430 reap_dead start 2024-04-16T15:40:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.339+0000 7ffadcd99640 1 -- 172.21.15.71:0/3841918430 shutdown_connections 2024-04-16T15:40:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.339+0000 7ffadcd99640 1 -- 172.21.15.71:0/3841918430 >> 172.21.15.71:0/3841918430 conn(0x7ffae80c7f50 msgr2=0x7ffae81591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.339+0000 7ffadcd99640 1 -- 172.21.15.71:0/3841918430 shutdown_connections 2024-04-16T15:40:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.339+0000 7ffadcd99640 1 -- 172.21.15.71:0/3841918430 wait complete. 2024-04-16T15:40:43.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.340+0000 7fc839ffb640 1 -- 172.21.15.71:0/1325571056 <== osd.1 v2:172.21.15.71:6808/357157073 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc820002dd0 con 0x7fc828076140 2024-04-16T15:40:43.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.340+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fc828076140 msgr2=0x7fc828078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.340+0000 7fc8277fe640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fc828076140 0x7fc828078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc82806c9c0 msgr2=0x7fc82806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fc8277fe640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc82806c9c0 0x7fc82806ee80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc834006fd0 tx=0x7fc834008040 comp rx=0 tx=0).stop 2024-04-16T15:40:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8441ebae0 msgr2=0x7fc844155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fc8277fe640 1 --2- 172.21.15.71:0/1325571056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8441ebae0 0x7fc844155390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc8400343c0 tx=0x7fc8400036f0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fc8498ef640 1 -- 172.21.15.71:0/1325571056 reap_dead start 2024-04-16T15:40:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 shutdown_connections 2024-04-16T15:40:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 >> 172.21.15.71:0/1325571056 conn(0x7fc8440a9e20 msgr2=0x7fc844073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.340+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 --> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbadc002dd0 con 0x7fbae8076ff0 2024-04-16T15:40:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fbae3fff640 1 -- 172.21.15.71:0/3765572767 <== osd.2 v2:172.21.15.71:6813/3827429750 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbadc002dd0 con 0x7fbae8076ff0 2024-04-16T15:40:43.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7f83adf59640 1 -- 172.21.15.71:0/124617506 --> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83a80c9940 con 0x7f8394076130 2024-04-16T15:40:43.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 shutdown_connections 2024-04-16T15:40:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fbae8076ff0 msgr2=0x7fbae8079490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.341+0000 7fbb01c7e640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fbae8076ff0 0x7fbae8079490 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.342+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbae806cd20 msgr2=0x7fbae806f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.342+0000 7fbb01c7e640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fbae806cd20 0x7fbae806f1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400e1e0 tx=0x7fbaf400b040 comp rx=0 tx=0).stop 2024-04-16T15:40:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.342+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc06f550 msgr2=0x7fbafc06ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.342+0000 7fbb01c7e640 1 --2- 172.21.15.71:0/3765572767 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbafc06f550 0x7fbafc06ec90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0015be0 tx=0x7fbaf0013560 comp rx=0 tx=0).stop 2024-04-16T15:40:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.342+0000 7fc8277fe640 1 -- 172.21.15.71:0/1325571056 wait complete. 2024-04-16T15:40:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.342+0000 7f838ffff640 1 -- 172.21.15.71:0/124617506 <== osd.3 v2:172.21.15.71:6824/2052589418 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83a80c9940 con 0x7f8394076130 2024-04-16T15:40:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.342+0000 7fbafbfff640 1 -- 172.21.15.71:0/3765572767 reap_dead start 2024-04-16T15:40:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 shutdown_connections 2024-04-16T15:40:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 >> 172.21.15.71:0/3765572767 conn(0x7fbafc0c7560 msgr2=0x7fbafc0c24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 -- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f8394076130 msgr2=0x7f8394078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f8394076130 0x7f8394078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 shutdown_connections 2024-04-16T15:40:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 -- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f839406c9c0 msgr2=0x7f839406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7fbb01c7e640 1 -- 172.21.15.71:0/3765572767 wait complete. 2024-04-16T15:40:43.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 --> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53700c3090 con 0x7f5354076070 2024-04-16T15:40:43.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 --> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2118068be0 con 0x7f20fc076130 2024-04-16T15:40:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f839406c9c0 0x7f839406ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8398009a80 tx=0x7f8398008040 comp rx=0 tx=0).stop 2024-04-16T15:40:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 -- 172.21.15.71:0/124617506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a80c93c0 msgr2=0x7f83a80c5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 --2- 172.21.15.71:0/124617506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83a80c93c0 0x7f83a80c5100 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f83a80cc440 tx=0x7f83a0004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f83a7fff640 1 -- 172.21.15.71:0/124617506 reap_dead start 2024-04-16T15:40:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 -- 172.21.15.71:0/124617506 shutdown_connections 2024-04-16T15:40:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 -- 172.21.15.71:0/124617506 >> 172.21.15.71:0/124617506 conn(0x7f83a806bb10 msgr2=0x7f83a80ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.343+0000 7f838dffb640 1 -- 172.21.15.71:0/124617506 shutdown_connections 2024-04-16T15:40:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f838dffb640 1 -- 172.21.15.71:0/124617506 wait complete. 2024-04-16T15:40:43.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f5366ffd640 1 -- 172.21.15.71:0/944734986 <== osd.4 v2:172.21.15.122:6816/710391503 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f53700c3090 con 0x7f5354076070 2024-04-16T15:40:43.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f21157fa640 1 -- 172.21.15.71:0/542549399 <== osd.0 v2:172.21.15.71:6800/2582424538 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2118068be0 con 0x7f20fc076130 2024-04-16T15:40:43.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 --> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec780057d0 con 0x7fec64076130 2024-04-16T15:40:43.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f5354076070 msgr2=0x7f53540784b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f5377277640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f5354076070 0x7f53540784b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7fec767fc640 1 -- 172.21.15.71:0/632809661 <== osd.5 v2:172.21.15.122:6817/1133463708 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec780057d0 con 0x7fec64076130 2024-04-16T15:40:43.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7f20fc076130 msgr2=0x7f20fc078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f211eda8640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7f20fc076130 0x7f20fc078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20fc06c9c0 msgr2=0x7f20fc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f211eda8640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20fc06c9c0 0x7f20fc06ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f210c02c9c0 tx=0x7f210c00a000 comp rx=0 tx=0).stop 2024-04-16T15:40:43.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21181522b0 msgr2=0x7f2118144270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.344+0000 7f211eda8640 1 --2- 172.21.15.71:0/542549399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21181522b0 0x7f2118144270 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f211000be20 tx=0x7f211000bf10 comp rx=0 tx=0).stop 2024-04-16T15:40:43.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f211d345640 1 -- 172.21.15.71:0/542549399 reap_dead start 2024-04-16T15:40:43.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 shutdown_connections 2024-04-16T15:40:43.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 >> 172.21.15.71:0/542549399 conn(0x7f21180c7560 msgr2=0x7f21180c33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f535406c970 msgr2=0x7f535406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5377277640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f535406c970 0x7f535406ee30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f537014cc20 tx=0x7f5368009040 comp rx=0 tx=0).stop 2024-04-16T15:40:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53700c34c0 msgr2=0x7f53701f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5377277640 1 --2- 172.21.15.71:0/944734986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53700c34c0 0x7f53701f2ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f536c000c00 tx=0x7f536c004730 comp rx=0 tx=0).stop 2024-04-16T15:40:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5376a76640 1 -- 172.21.15.71:0/944734986 reap_dead start 2024-04-16T15:40:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 shutdown_connections 2024-04-16T15:40:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 >> 172.21.15.71:0/944734986 conn(0x7f53700c7480 msgr2=0x7f53700c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7fec64076130 msgr2=0x7fec64078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7fec87d50640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7fec64076130 0x7fec64078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 shutdown_connections 2024-04-16T15:40:43.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f5377277640 1 -- 172.21.15.71:0/944734986 wait complete. 2024-04-16T15:40:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 shutdown_connections 2024-04-16T15:40:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7f211eda8640 1 -- 172.21.15.71:0/542549399 wait complete. 2024-04-16T15:40:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec6406c9c0 msgr2=0x7fec6406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.345+0000 7fec87d50640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec6406c9c0 0x7fec6406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec70036000 tx=0x7fec7003c040 comp rx=0 tx=0).stop 2024-04-16T15:40:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.346+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec780dbfe0 msgr2=0x7fec78175030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.346+0000 7fec87d50640 1 --2- 172.21.15.71:0/632809661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec780dbfe0 0x7fec78175030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec8006a920 tx=0x7fec80075650 comp rx=0 tx=0).stop 2024-04-16T15:40:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.346+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 --> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e94070330 con 0x7f5e74076170 2024-04-16T15:40:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.347+0000 7fec862ed640 1 -- 172.21.15.71:0/632809661 reap_dead start 2024-04-16T15:40:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.347+0000 7f5e82ffd640 1 -- 172.21.15.71:0/3638336677 <== osd.7 v2:172.21.15.122:6800/3651959178 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5e94070330 con 0x7f5e74076170 2024-04-16T15:40:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.347+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 shutdown_connections 2024-04-16T15:40:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.347+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 >> 172.21.15.71:0/632809661 conn(0x7fec780062d0 msgr2=0x7fec7802b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.347+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f5e74076170 msgr2=0x7f5e740785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.347+0000 7f5e98d59640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f5e74076170 0x7f5e740785b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5e7406c9f0 msgr2=0x7f5e7406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e98d59640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5e7406c9f0 0x7f5e7406eeb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e94150320 tx=0x7f5e88007e30 comp rx=0 tx=0).stop 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e940ae220 msgr2=0x7f5e941f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e98d59640 1 --2- 172.21.15.71:0/3638336677 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5e940ae220 0x7f5e941f3f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000f80 tx=0x7f5e8c004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e92d9d640 1 -- 172.21.15.71:0/3638336677 reap_dead start 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 shutdown_connections 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 >> 172.21.15.71:0/3638336677 conn(0x7f5e9406f800 msgr2=0x7f5e940a9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 shutdown_connections 2024-04-16T15:40:43.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7f5e98d59640 1 -- 172.21.15.71:0/3638336677 wait complete. 2024-04-16T15:40:43.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 shutdown_connections 2024-04-16T15:40:43.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.348+0000 7fec87d50640 1 -- 172.21.15.71:0/632809661 wait complete. 2024-04-16T15:40:43.408 INFO:teuthology.orchestra.run.smithi071.stdout:55834574859 2024-04-16T15:40:43.408 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-16T15:40:43.409 INFO:teuthology.orchestra.run.smithi071.stdout:55834574858 2024-04-16T15:40:43.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-16T15:40:43.410 INFO:teuthology.orchestra.run.smithi071.stdout:55834574859 2024-04-16T15:40:43.410 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-16T15:40:43.411 INFO:teuthology.orchestra.run.smithi071.stdout:55834574857 2024-04-16T15:40:43.411 INFO:teuthology.orchestra.run.smithi071.stdout:55834574858 2024-04-16T15:40:43.412 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-16T15:40:43.412 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-16T15:40:43.413 INFO:teuthology.orchestra.run.smithi071.stdout:55834574858 2024-04-16T15:40:43.413 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-16T15:40:43.414 INFO:teuthology.orchestra.run.smithi071.stdout:55834574859 2024-04-16T15:40:43.414 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-16T15:40:43.415 INFO:teuthology.orchestra.run.smithi071.stdout:55834574857 2024-04-16T15:40:43.415 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-16T15:40:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.624+0000 7feb2c567640 1 Processor -- start 2024-04-16T15:40:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.625+0000 7fad7e759640 1 Processor -- start 2024-04-16T15:40:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.624+0000 7feb2c567640 1 -- start start 2024-04-16T15:40:43.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7fad7e759640 1 -- start start 2024-04-16T15:40:43.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7f63e1d8f640 1 Processor -- start 2024-04-16T15:40:43.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7feb2c567640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7feb24157be0 0x7feb240c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7feb2c567640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb240c62d0 0x7feb240c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7feb2c567640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb240701d0 con 0x7feb24156f70 2024-04-16T15:40:43.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7feb2c567640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7feb241514e0 con 0x7feb24157be0 2024-04-16T15:40:43.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7feb2c567640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feb2406fc10 con 0x7feb240c62d0 2024-04-16T15:40:43.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7feb2ab04640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb240c62d0 0x7feb240c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7feb2ab04640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb240c62d0 0x7feb240c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34226/0 (socket says 172.21.15.71:34226) 2024-04-16T15:40:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7feb2ab04640 1 -- 172.21.15.71:0/1632353655 learned_addr learned my addr 172.21.15.71:0/1632353655 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7fad7e759640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fad78157700 0x7fad7815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7fdde0e77640 1 Processor -- start 2024-04-16T15:40:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7feb2ab04640 1 -- 172.21.15.71:0/1632353655 >> v2:172.21.15.71:3300/0 conn(0x7feb24157be0 msgr2=0x7feb240c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7feb2ab04640 1 --2- 172.21.15.71:0/1632353655 >> v2:172.21.15.71:3300/0 conn(0x7feb24157be0 0x7feb240c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7feb2ab04640 1 -- 172.21.15.71:0/1632353655 >> v1:172.21.15.71:6789/0 conn(0x7feb24156f70 legacy=0x7feb24154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7feb2ab04640 1 -- 172.21.15.71:0/1632353655 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2406fef0 con 0x7feb240c62d0 2024-04-16T15:40:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7f2dc45ba640 1 Processor -- start 2024-04-16T15:40:43.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7f2dc45ba640 1 -- start start 2024-04-16T15:40:43.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7f501efab640 1 Processor -- start 2024-04-16T15:40:43.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7f63e1d8f640 1 -- start start 2024-04-16T15:40:43.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7f63e1d8f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f63dc073200 0x7f63dc070220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7f63e1d8f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 0x7f63dc0a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7f63e1d8f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63dc072160 con 0x7f63dc06fe40 2024-04-16T15:40:43.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7f63e1d8f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f63dc073660 con 0x7f63dc073200 2024-04-16T15:40:43.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7f63e1d8f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc072d70 con 0x7f63dc0a7ca0 2024-04-16T15:40:43.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7fd3e41a8640 1 Processor -- start 2024-04-16T15:40:43.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7fd3e41a8640 1 -- start start 2024-04-16T15:40:43.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7f501efab640 1 -- start start 2024-04-16T15:40:43.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7fad7e759640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad78072550 con 0x7fad780c4cc0 2024-04-16T15:40:43.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7fad7e759640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad780c51c0 con 0x7fad780ac670 2024-04-16T15:40:43.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.627+0000 7fad7e759640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fad7815af90 con 0x7fad78157700 2024-04-16T15:40:43.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.626+0000 7fdde0e77640 1 -- start start 2024-04-16T15:40:43.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63daffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f63dc073200 0x7f63dc070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63dbfff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 0x7f63dc0a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63daffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f63dc073200 0x7f63dc070220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51158/0 (socket says 172.21.15.71:51158) 2024-04-16T15:40:43.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63daffd640 1 -- 172.21.15.71:0/1746489093 learned_addr learned my addr 172.21.15.71:0/1746489093 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f37363cd640 1 Processor -- start 2024-04-16T15:40:43.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fdddb7fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fddd40dbce0 0x7fddd40df180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:60762/0 (socket says 172.21.15.71:60762) 2024-04-16T15:40:43.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fdddaffd640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fddd40474a0 0x7fddd4006ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34232/0 (socket says 172.21.15.71:34232) 2024-04-16T15:40:43.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f2dc45ba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2dbc071df0 0x7f2dbc0c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501efab640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50100071b0 0x7f5010007590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501efab640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f501004db50 0x7f50100e2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501efab640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f501000e9c0 0x7f501000ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501efab640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f501000e530 con 0x7f501000e9c0 2024-04-16T15:40:43.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501efab640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f50100e2c70 con 0x7f501004db50 2024-04-16T15:40:43.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501efab640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f501004a560 con 0x7f50100071b0 2024-04-16T15:40:43.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fdddb7fe640 1 -- 172.21.15.71:0/1369378281 learned_addr learned my addr 172.21.15.71:0/1369378281 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:43.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fdde0e77640 1 --2- 172.21.15.71:0/1369378281 >> v2:172.21.15.71:3300/0 conn(0x7fddd400a8b0 0x7fddd400ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63da7fc640 1 -- 172.21.15.71:0/1746489093 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603281001 0 0) 0x7f63dc072160 con 0x7f63dc06fe40 2024-04-16T15:40:43.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63da7fc640 1 -- 172.21.15.71:0/1746489093 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63c8003520 con 0x7f63dc06fe40 2024-04-16T15:40:43.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fd3e41a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc0c93c0 0x7fd3dc06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fd3e41a8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fd3dc0beb50 0x7fd3dc06d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fd3e41a8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3dc156880 con 0x7fd3dc14c730 2024-04-16T15:40:43.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fd3e41a8640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fd3dc156b60 con 0x7fd3dc0beb50 2024-04-16T15:40:43.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fd3e41a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd3dc156630 con 0x7fd3dc0c93c0 2024-04-16T15:40:43.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63daffd640 1 -- 172.21.15.71:0/1746489093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 msgr2=0x7f63dc0a8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63daffd640 1 --2- 172.21.15.71:0/1746489093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 0x7f63dc0a8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63daffd640 1 -- 172.21.15.71:0/1746489093 >> v1:172.21.15.122:6789/0 conn(0x7f63dc06fe40 legacy=0x7f63dc15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f63daffd640 1 -- 172.21.15.71:0/1746489093 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc072440 con 0x7f63dc073200 2024-04-16T15:40:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501d7a8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f501004db50 0x7f50100e2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501d7a8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f501004db50 0x7f50100e2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51176/0 (socket says 172.21.15.71:51176) 2024-04-16T15:40:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f501d7a8640 1 -- 172.21.15.71:0/4286713821 learned_addr learned my addr 172.21.15.71:0/4286713821 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f2dc2356640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f2dbc14f0e0 0x7f2dbc0c2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:60766/0 (socket says 172.21.15.71:60766) 2024-04-16T15:40:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f501dfa9640 1 --2- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50100071b0 0x7f5010007590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fd3e2745640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fd3dc0beb50 0x7fd3dc06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7feb2ab04640 1 --2- 172.21.15.71:0/1632353655 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb240c62d0 0x7feb240c6760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb14009870 tx=0x7feb1402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5905ac75a2e59e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f2dc2356640 1 -- 172.21.15.71:0/3997927921 learned_addr learned my addr 172.21.15.71:0/3997927921 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f2dc1b55640 1 --2- 172.21.15.71:0/3997927921 >> v2:172.21.15.122:3300/0 conn(0x7f2dbc071df0 0x7f2dbc0c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fdddbfff640 1 --2- 172.21.15.71:0/1369378281 >> v2:172.21.15.71:3300/0 conn(0x7fddd400a8b0 0x7fddd400ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fad7ccf6640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fad780ac670 0x7fad780acae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:60760/0 (socket says 172.21.15.71:60760) 2024-04-16T15:40:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fad7ccf6640 1 -- 172.21.15.71:0/3882031749 learned_addr learned my addr 172.21.15.71:0/3882031749 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fad77fff640 1 --2- 172.21.15.71:0/3882031749 >> v2:172.21.15.122:3300/0 conn(0x7fad78157700 0x7fad7815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fad77fff640 1 -- 172.21.15.71:0/3882031749 >> v1:172.21.15.122:6789/0 conn(0x7fad780ac670 legacy=0x7fad780acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fad77fff640 1 -- 172.21.15.71:0/3882031749 >> v1:172.21.15.71:6789/0 conn(0x7fad780c4cc0 legacy=0x7fad780ad720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T15:40:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7fad77fff640 1 -- 172.21.15.71:0/3882031749 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad780a8800 con 0x7fad78157700 2024-04-16T15:40:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fdde0e77640 1 -- 172.21.15.71:0/1369378281 --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd400bb50 con 0x7fddd40474a0 2024-04-16T15:40:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fdde0e77640 1 -- 172.21.15.71:0/1369378281 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd4048060 con 0x7fddd40dbce0 2024-04-16T15:40:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fdde0e77640 1 -- 172.21.15.71:0/1369378281 --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fddd400a420 con 0x7fddd400a8b0 2024-04-16T15:40:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2054948688 0 0) 0x7fddd4048060 con 0x7fddd40dbce0 2024-04-16T15:40:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddcc003520 con 0x7fddd40dbce0 2024-04-16T15:40:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2033497414 0 0) 0x7fddd400bb50 con 0x7fddd40474a0 2024-04-16T15:40:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddd4048060 con 0x7fddd40474a0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7feb29301640 1 -- 172.21.15.71:0/1632353655 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140317c0 con 0x7feb240c62d0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7feb29301640 1 -- 172.21.15.71:0/1632353655 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb14037070 con 0x7feb240c62d0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7feb29301640 1 -- 172.21.15.71:0/1632353655 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14031b80 con 0x7feb240c62d0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f2dc45ba640 1 -- 172.21.15.71:0/3997927921 --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dbc0c2090 con 0x7f2dbc1525f0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f2dc45ba640 1 -- 172.21.15.71:0/3997927921 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dbc0c3b90 con 0x7f2dbc14f0e0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f2dc45ba640 1 -- 172.21.15.71:0/3997927921 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2dbc0a7540 con 0x7f2dbc071df0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 940425173 0 0) 0x7f2dbc0c3b90 con 0x7f2dbc14f0e0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2da8003520 con 0x7f2dbc14f0e0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1042533134 0 0) 0x7f2da8003520 con 0x7f2dbc14f0e0 2024-04-16T15:40:43.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 >> v2:172.21.15.122:3300/0 conn(0x7f2dbc071df0 msgr2=0x7f2dbc0c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 --2- 172.21.15.71:0/3997927921 >> v2:172.21.15.122:3300/0 conn(0x7f2dbc071df0 0x7f2dbc0c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 >> v1:172.21.15.71:6789/0 conn(0x7f2dbc1525f0 legacy=0x7f2dbc06fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc0c3e70 con 0x7f2dbc14f0e0 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f2db40019f0 con 0x7f2dbc14f0e0 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 >> v1:172.21.15.122:6789/0 conn(0x7f2dbc14f0e0 legacy=0x7f2dbc0c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f501e7aa640 1 --2- 172.21.15.71:0/4286713821 >> v2:172.21.15.71:3300/0 conn(0x7f501000e9c0 0x7f501000ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f501d7a8640 1 -- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50100071b0 msgr2=0x7f5010007590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f501d7a8640 1 --2- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50100071b0 0x7f5010007590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f501d7a8640 1 -- 172.21.15.71:0/4286713821 >> v2:172.21.15.71:3300/0 conn(0x7f501000e9c0 msgr2=0x7f501000ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f501d7a8640 1 --2- 172.21.15.71:0/4286713821 >> v2:172.21.15.71:3300/0 conn(0x7f501000e9c0 0x7f501000ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f501d7a8640 1 -- 172.21.15.71:0/4286713821 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501000e200 con 0x7f501004db50 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fd3e1f44640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fd3dc14c730 0x7fd3dc0c8650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34248/0 (socket says 172.21.15.71:34248) 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fd3e1f44640 1 -- 172.21.15.71:0/1300448787 learned_addr learned my addr 172.21.15.71:0/1300448787 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7fd3e1743640 1 --2- 172.21.15.71:0/1300448787 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc0c93c0 0x7fd3dc06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fd3e0f42640 1 -- 172.21.15.71:0/1300448787 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1011599562 0 0) 0x7fd3dc156880 con 0x7fd3dc14c730 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fd3e0f42640 1 -- 172.21.15.71:0/1300448787 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3c8003520 con 0x7fd3dc14c730 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fd3e1743640 1 -- 172.21.15.71:0/1300448787 >> v2:172.21.15.71:3300/0 conn(0x7fd3dc0beb50 msgr2=0x7fd3dc06d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fd3e1743640 1 --2- 172.21.15.71:0/1300448787 >> v2:172.21.15.71:3300/0 conn(0x7fd3dc0beb50 0x7fd3dc06d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fd3e1743640 1 -- 172.21.15.71:0/1300448787 >> v1:172.21.15.71:6789/0 conn(0x7fd3dc14c730 legacy=0x7fd3dc0c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fd3e1743640 1 -- 172.21.15.71:0/1300448787 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc0c5fe0 con 0x7fd3dc0c93c0 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.628+0000 7f37363cd640 1 -- start start 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f37363cd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c1f80 0x7f37300ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.629+0000 7f37363cd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f37300c7ea0 0x7f3730151c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f37363cd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37301560e0 con 0x7f3730070000 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f37363cd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f3730155260 con 0x7f37300c7ea0 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f37363cd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37300aa4f0 con 0x7f37300c1f80 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f372ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c1f80 0x7f37300ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f372ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c1f80 0x7f37300ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34282/0 (socket says 172.21.15.71:34282) 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f372ffff640 1 -- 172.21.15.71:0/3934282553 learned_addr learned my addr 172.21.15.71:0/3934282553 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f373496a640 1 --2- 172.21.15.71:0/3934282553 >> v2:172.21.15.122:3300/0 conn(0x7f37300c7ea0 0x7f3730151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2da80037a0 0x7f2da8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 381663089 0 0) 0x7fddcc003520 con 0x7fddd40dbce0 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 >> v2:172.21.15.71:3300/0 conn(0x7fddd400a8b0 msgr2=0x7fddd400ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 --2- 172.21.15.71:0/1369378281 >> v2:172.21.15.71:3300/0 conn(0x7fddd400a8b0 0x7fddd400ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 >> v1:172.21.15.71:6789/0 conn(0x7fddd40474a0 legacy=0x7fddd4006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd400a0f0 con 0x7fddd40dbce0 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fddd0001670 con 0x7fddd40dbce0 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 >> v1:172.21.15.122:6789/0 conn(0x7fddd40dbce0 legacy=0x7fddd40df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddcc003be0 0x7fddcc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddcc004800 0x7fddcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc0052e0 0x7fddcc00ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fdddb7fe640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddcc003be0 0x7fddcc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fdddbfff640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc0052e0 0x7fddcc00ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fddd400a420 con 0x7fddcc0052e0 2024-04-16T15:40:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fddcc01d2c0 con 0x7fddcc003be0 2024-04-16T15:40:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fddcc01d4c0 con 0x7fddcc004800 2024-04-16T15:40:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7feb2c567640 1 -- 172.21.15.71:0/1632353655 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb240c62d0 msgr2=0x7feb240c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7feb2c567640 1 --2- 172.21.15.71:0/1632353655 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb240c62d0 0x7feb240c6760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb14009870 tx=0x7feb1402b860 comp rx=0 tx=0).stop 2024-04-16T15:40:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da80040c0 0x7f2da800b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2da800b600 0x7f2da800daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fdddaffd640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddcc004800 0x7fddcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7fad77fff640 1 --2- 172.21.15.71:0/3882031749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad78157700 0x7fad7815aa50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad70009d90 tx=0x7fad7002be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724cbe15cf75f1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fad76ffd640 1 -- 172.21.15.71:0/3882031749 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70039070 con 0x7fad78157700 2024-04-16T15:40:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fad76ffd640 1 -- 172.21.15.71:0/3882031749 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7002cbd0 con 0x7fad78157700 2024-04-16T15:40:43.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fad76ffd640 1 -- 172.21.15.71:0/3882031749 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70034440 con 0x7fad78157700 2024-04-16T15:40:43.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fdddb7fe640 1 -- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddcc004800 msgr2=0x7fddcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fdddb7fe640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddcc004800 0x7fddcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fdddb7fe640 1 -- 172.21.15.71:0/1369378281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc0052e0 msgr2=0x7fddcc00ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fdddb7fe640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc0052e0 0x7fddcc00ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fdddb7fe640 1 -- 172.21.15.71:0/1369378281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc003520 con 0x7fddcc003be0 2024-04-16T15:40:43.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7feb2c567640 1 -- 172.21.15.71:0/1632353655 shutdown_connections 2024-04-16T15:40:43.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7feb2c567640 1 --2- 172.21.15.71:0/1632353655 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb240c62d0 0x7feb240c6760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7feb2c567640 1 --2- 172.21.15.71:0/1632353655 >> v2:172.21.15.71:3300/0 conn(0x7feb24157be0 0x7feb240c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7feb2c567640 1 -- 172.21.15.71:0/1632353655 >> 172.21.15.71:0/1632353655 conn(0x7feb240ab440 msgr2=0x7feb240ab850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7feb2c567640 1 -- 172.21.15.71:0/1632353655 shutdown_connections 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0a7540 con 0x7f2da80037a0 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2da800e380 con 0x7f2da800b600 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc1b55640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da80040c0 0x7f2da800b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2da800e580 con 0x7f2da80040c0 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc2b57640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2da800b600 0x7f2da800daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc2356640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2da80037a0 0x7f2da8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f501dfa9640 1 --2- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50100071b0 0x7f5010007590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:43.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f501e7aa640 1 --2- 172.21.15.71:0/4286713821 >> v2:172.21.15.71:3300/0 conn(0x7f501000e9c0 0x7f501000ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:43.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f501d7a8640 1 --2- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501004db50 0x7f50100e2730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f501004d760 tx=0x7f500802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a542fa8f0bb0793 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f63daffd640 1 --2- 172.21.15.71:0/1746489093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 0x7f63dc070220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009e50 tx=0x7f63cc02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f85c76a51b8ef56 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f372ffff640 1 -- 172.21.15.71:0/3934282553 >> v2:172.21.15.122:3300/0 conn(0x7f37300c7ea0 msgr2=0x7f3730151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f372ffff640 1 --2- 172.21.15.71:0/3934282553 >> v2:172.21.15.122:3300/0 conn(0x7f37300c7ea0 0x7f3730151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f372ffff640 1 -- 172.21.15.71:0/3934282553 >> v1:172.21.15.122:6789/0 conn(0x7f3730070000 legacy=0x7f37301559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.630+0000 7f372ffff640 1 -- 172.21.15.71:0/3934282553 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37300aa7d0 con 0x7f37300c1f80 2024-04-16T15:40:43.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7feb2c567640 1 -- 172.21.15.71:0/1632353655 wait complete. 2024-04-16T15:40:43.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fdddbfff640 1 -- 172.21.15.71:0/1369378281 reap_dead start 2024-04-16T15:40:43.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc2b57640 1 -- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da80040c0 msgr2=0x7f2da800b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f501cfa7640 1 -- 172.21.15.71:0/4286713821 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008034070 con 0x7f501004db50 2024-04-16T15:40:43.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f501cfa7640 1 -- 172.21.15.71:0/4286713821 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5008030610 con 0x7f501004db50 2024-04-16T15:40:43.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f501cfa7640 1 -- 172.21.15.71:0/4286713821 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008030a20 con 0x7f501004db50 2024-04-16T15:40:43.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f63da7fc640 1 -- 172.21.15.71:0/1746489093 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc02cb00 con 0x7f63dc073200 2024-04-16T15:40:43.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f63da7fc640 1 -- 172.21.15.71:0/1746489093 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc02cca0 con 0x7f63dc073200 2024-04-16T15:40:43.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f63da7fc640 1 -- 172.21.15.71:0/1746489093 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc034610 con 0x7f63dc073200 2024-04-16T15:40:43.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7fd3e1743640 1 --2- 172.21.15.71:0/1300448787 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc0c93c0 0x7fd3dc06cfc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0097a0 tx=0x7fd3cc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47fcce512ca866da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc2b57640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da80040c0 0x7f2da800b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc2b57640 1 -- 172.21.15.71:0/3997927921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2da80037a0 msgr2=0x7f2da8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:43.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc2b57640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2da80037a0 0x7f2da8003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.631+0000 7f2dc2b57640 1 -- 172.21.15.71:0/3997927921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da8003520 con 0x7f2da800b600 2024-04-16T15:40:43.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fdddb7fe640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddcc003be0 0x7fddcc003fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fddd0001eb0 tx=0x7fddcc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da41aa6e31353fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f372ffff640 1 --2- 172.21.15.71:0/3934282553 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c1f80 0x7f37300ac210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f37301553e0 tx=0x7f372802bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=366863a02528675b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fad7e759640 1 -- 172.21.15.71:0/3882031749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad78157700 msgr2=0x7fad7815aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fad7e759640 1 --2- 172.21.15.71:0/3882031749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad78157700 0x7fad7815aa50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad70009d90 tx=0x7fad7002be20 comp rx=0 tx=0).stop 2024-04-16T15:40:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fad7e759640 1 -- 172.21.15.71:0/3882031749 shutdown_connections 2024-04-16T15:40:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fad7e759640 1 --2- 172.21.15.71:0/3882031749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad78157700 0x7fad7815aa50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fad7e759640 1 -- 172.21.15.71:0/3882031749 >> 172.21.15.71:0/3882031749 conn(0x7fad780ab270 msgr2=0x7fad780ab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fd3e0f42640 1 -- 172.21.15.71:0/1300448787 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc035020 con 0x7fd3dc0c93c0 2024-04-16T15:40:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fd3e0f42640 1 -- 172.21.15.71:0/1300448787 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3cc031830 con 0x7fd3dc0c93c0 2024-04-16T15:40:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fd3e0f42640 1 -- 172.21.15.71:0/1300448787 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc031c00 con 0x7fd3dc0c93c0 2024-04-16T15:40:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0007c40 con 0x7fddcc003be0 2024-04-16T15:40:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd0030070 con 0x7fddcc003be0 2024-04-16T15:40:43.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fddda7fc640 1 -- 172.21.15.71:0/1369378281 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0005dd0 con 0x7fddcc003be0 2024-04-16T15:40:43.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f2dc2b57640 1 -- 172.21.15.71:0/3997927921 reap_dead start 2024-04-16T15:40:43.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f501efab640 1 -- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501004db50 msgr2=0x7f50100e2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f501efab640 1 --2- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501004db50 0x7f50100e2730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f501004d760 tx=0x7f500802b960 comp rx=0 tx=0).stop 2024-04-16T15:40:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f372effd640 1 -- 172.21.15.71:0/3934282553 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372802c930 con 0x7f37300c1f80 2024-04-16T15:40:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f372effd640 1 -- 172.21.15.71:0/3934282553 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372802cad0 con 0x7f37300c1f80 2024-04-16T15:40:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f372effd640 1 -- 172.21.15.71:0/3934282553 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37280343f0 con 0x7f37300c1f80 2024-04-16T15:40:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fad7e759640 1 -- 172.21.15.71:0/3882031749 shutdown_connections 2024-04-16T15:40:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f501efab640 1 -- 172.21.15.71:0/4286713821 shutdown_connections 2024-04-16T15:40:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f501efab640 1 --2- 172.21.15.71:0/4286713821 >> v2:172.21.15.71:3300/0 conn(0x7f501000e9c0 0x7f501000ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f501efab640 1 --2- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501004db50 0x7f50100e2730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f501efab640 1 --2- 172.21.15.71:0/4286713821 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f50100071b0 0x7f5010007590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f63e1d8f640 1 -- 172.21.15.71:0/1746489093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 msgr2=0x7f63dc070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f501efab640 1 -- 172.21.15.71:0/4286713821 >> 172.21.15.71:0/4286713821 conn(0x7f501004c660 msgr2=0x7f501004ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7fad7e759640 1 -- 172.21.15.71:0/3882031749 wait complete. 2024-04-16T15:40:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/1746489093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 0x7f63dc070220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009e50 tx=0x7f63cc02bf60 comp rx=0 tx=0).stop 2024-04-16T15:40:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f501efab640 1 -- 172.21.15.71:0/4286713821 shutdown_connections 2024-04-16T15:40:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.632+0000 7f63e1d8f640 1 -- 172.21.15.71:0/1746489093 shutdown_connections 2024-04-16T15:40:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/1746489093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 0x7f63dc0a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/1746489093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 0x7f63dc070220 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f63e1d8f640 1 -- 172.21.15.71:0/1746489093 >> 172.21.15.71:0/1746489093 conn(0x7f63dc06b930 msgr2=0x7f63dc06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7feb2c567640 1 Processor -- start 2024-04-16T15:40:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f2dc2b57640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2da800b600 0x7f2da800daa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0c8750 tx=0x7f2da8004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d247523947db524 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fdde0e77640 1 -- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddcc003be0 msgr2=0x7fddcc003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fdde0e77640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddcc003be0 0x7fddcc003fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fddd0001eb0 tx=0x7fddcc005830 comp rx=0 tx=0).stop 2024-04-16T15:40:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac001370 con 0x7f2da800b600 2024-04-16T15:40:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac0052d0 con 0x7f2da800b600 2024-04-16T15:40:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f2dc1354640 1 -- 172.21.15.71:0/3997927921 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac003830 con 0x7f2da800b600 2024-04-16T15:40:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f63e1d8f640 1 -- 172.21.15.71:0/1746489093 shutdown_connections 2024-04-16T15:40:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fd3e41a8640 1 -- 172.21.15.71:0/1300448787 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc0c93c0 msgr2=0x7fd3dc06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fdde0e77640 1 -- 172.21.15.71:0/1369378281 shutdown_connections 2024-04-16T15:40:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fdde0e77640 1 --2- 172.21.15.71:0/1369378281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddcc003be0 0x7fddcc003fe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fdde0e77640 1 -- 172.21.15.71:0/1369378281 >> 172.21.15.71:0/1369378281 conn(0x7fddd4048250 msgr2=0x7fddd4048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f37363cd640 1 -- 172.21.15.71:0/3934282553 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c1f80 msgr2=0x7f37300ac210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f37363cd640 1 --2- 172.21.15.71:0/3934282553 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c1f80 0x7f37300ac210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f37301553e0 tx=0x7f372802bd90 comp rx=0 tx=0).stop 2024-04-16T15:40:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fdde0e77640 1 -- 172.21.15.71:0/1369378281 shutdown_connections 2024-04-16T15:40:43.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/1300448787 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc0c93c0 0x7fd3dc06cfc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0097a0 tx=0x7fd3cc02b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f37363cd640 1 -- 172.21.15.71:0/3934282553 shutdown_connections 2024-04-16T15:40:43.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f37363cd640 1 --2- 172.21.15.71:0/3934282553 >> v2:172.21.15.122:3300/0 conn(0x7f37300c7ea0 0x7f3730151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f37363cd640 1 --2- 172.21.15.71:0/3934282553 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c1f80 0x7f37300ac210 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f37363cd640 1 -- 172.21.15.71:0/3934282553 >> 172.21.15.71:0/3934282553 conn(0x7f37300c7310 msgr2=0x7f37300c76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f37363cd640 1 -- 172.21.15.71:0/3934282553 shutdown_connections 2024-04-16T15:40:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fd3e41a8640 1 -- 172.21.15.71:0/1300448787 shutdown_connections 2024-04-16T15:40:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/1300448787 >> v2:172.21.15.71:3300/0 conn(0x7fd3dc0beb50 0x7fd3dc06d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/1300448787 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc0c93c0 0x7fd3dc06cfc0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0097a0 tx=0x7fd3cc02b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fd3e41a8640 1 -- 172.21.15.71:0/1300448787 >> 172.21.15.71:0/1300448787 conn(0x7fd3dc0ad5e0 msgr2=0x7fd3dc0c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7fd3e41a8640 1 -- 172.21.15.71:0/1300448787 shutdown_connections 2024-04-16T15:40:43.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7fdde0e77640 1 -- 172.21.15.71:0/1369378281 wait complete. 2024-04-16T15:40:43.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7f37363cd640 1 -- 172.21.15.71:0/3934282553 wait complete. 2024-04-16T15:40:43.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7f2dc45ba640 1 -- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2da800b600 msgr2=0x7f2da800daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7fad7e759640 1 Processor -- start 2024-04-16T15:40:43.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2da800b600 0x7f2da800daa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0c8750 tx=0x7f2da8004820 comp rx=0 tx=0).stop 2024-04-16T15:40:43.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7f63e1d8f640 1 -- 172.21.15.71:0/1746489093 wait complete. 2024-04-16T15:40:43.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.633+0000 7feb2c567640 1 -- start start 2024-04-16T15:40:43.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7feb2c567640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb240c62d0 0x7feb24151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7feb2c567640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24156f70 0x7feb241efed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7feb2c567640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb24157be0 0x7feb241f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7feb2c567640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb241573d0 con 0x7feb24156f70 2024-04-16T15:40:43.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7feb2c567640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb24153e60 con 0x7feb240c62d0 2024-04-16T15:40:43.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7feb2c567640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feb2406f830 con 0x7feb24157be0 2024-04-16T15:40:43.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7f2dc45ba640 1 -- 172.21.15.71:0/3997927921 shutdown_connections 2024-04-16T15:40:43.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/3997927921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2da800b600 0x7f2da800daa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7f2dc45ba640 1 -- 172.21.15.71:0/3997927921 >> 172.21.15.71:0/3997927921 conn(0x7f2dbc06b930 msgr2=0x7f2dbc06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:43.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7f63e1d8f640 1 Processor -- start 2024-04-16T15:40:43.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7fad7e759640 1 -- start start 2024-04-16T15:40:43.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7fdde0e77640 1 Processor -- start 2024-04-16T15:40:43.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7feb29b02640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24156f70 0x7feb241efed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7feb29b02640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24156f70 0x7feb241efed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46988/0 (socket says 172.21.15.71:46988) 2024-04-16T15:40:43.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7feb29b02640 1 -- 172.21.15.71:0/1860712752 learned_addr learned my addr 172.21.15.71:0/1860712752 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7feb2ab04640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb24157be0 0x7feb241f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7feb2a303640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb240c62d0 0x7feb24151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7fdde0e77640 1 -- start start 2024-04-16T15:40:43.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7f501efab640 1 -- 172.21.15.71:0/4286713821 wait complete. 2024-04-16T15:40:43.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7feb29b02640 1 -- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb24157be0 msgr2=0x7feb241f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7feb29b02640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb24157be0 0x7feb241f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7feb29b02640 1 -- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb240c62d0 msgr2=0x7feb24151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7feb29b02640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb240c62d0 0x7feb24151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7feb29b02640 1 -- 172.21.15.71:0/1860712752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb241f0950 con 0x7feb24156f70 2024-04-16T15:40:43.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.634+0000 7fd3e41a8640 1 -- 172.21.15.71:0/1300448787 wait complete. 2024-04-16T15:40:43.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7f2dc45ba640 1 -- 172.21.15.71:0/3997927921 shutdown_connections 2024-04-16T15:40:43.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7f2dc45ba640 1 -- 172.21.15.71:0/3997927921 wait complete. 2024-04-16T15:40:43.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7f37363cd640 1 Processor -- start 2024-04-16T15:40:43.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.635+0000 7f63e1d8f640 1 -- start start 2024-04-16T15:40:43.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fdde0e77640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc003be0 0x7fddd40d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7feb29b02640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24156f70 0x7feb241efed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c780 tx=0x7feb2000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7feb2ab04640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb24157be0 0x7feb241f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:43.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7feb1affd640 1 -- 172.21.15.71:0/1860712752 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20017070 con 0x7feb24156f70 2024-04-16T15:40:43.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7feb2c567640 1 -- 172.21.15.71:0/1860712752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb241525d0 con 0x7feb24156f70 2024-04-16T15:40:43.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7f501efab640 1 Processor -- start 2024-04-16T15:40:43.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7f63e1d8f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63dc06fe40 0x7f63dc1f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fd3e41a8640 1 Processor -- start 2024-04-16T15:40:43.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7f37363cd640 1 -- start start 2024-04-16T15:40:43.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fdde0e77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddd40428a0 0x7fddd416e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7feb2c567640 1 -- 172.21.15.71:0/1860712752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb241ef9d0 con 0x7feb24156f70 2024-04-16T15:40:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fad7e759640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad780ac670 0x7fad780be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fad7e759640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad780c4cc0 0x7fad781f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fad7e759640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad78157700 0x7fad781fa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fad7e759640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fad780ade30 con 0x7fad780c4cc0 2024-04-16T15:40:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fad7e759640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad780a8dc0 con 0x7fad78157700 2024-04-16T15:40:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fad7e759640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fad780adfb0 con 0x7fad780ac670 2024-04-16T15:40:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7fdde0e77640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddd416ecf0 0x7fddd40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fdde0e77640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fddd40dc0c0 con 0x7fddcc003be0 2024-04-16T15:40:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad780ac670 0x7fad780be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fdddaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddd40428a0 0x7fddd416e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fdddaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddd40428a0 0x7fddd416e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51240/0 (socket says 172.21.15.71:51240) 2024-04-16T15:40:43.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fdddaffd640 1 -- 172.21.15.71:0/3896869118 learned_addr learned my addr 172.21.15.71:0/3896869118 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad780ac670 0x7fad780be9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34314/0 (socket says 172.21.15.71:34314) 2024-04-16T15:40:43.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 -- 172.21.15.71:0/874518688 learned_addr learned my addr 172.21.15.71:0/874518688 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fdddbfff640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddd416ecf0 0x7fddd40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fddd4047d80 con 0x7fddd40428a0 2024-04-16T15:40:43.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fddd40dfbf0 con 0x7fddd416ecf0 2024-04-16T15:40:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad777fe640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad780c4cc0 0x7fad781f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7f37363cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730070000 0x7f37300c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7f37363cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37300c1f80 0x7f3730147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 -- 172.21.15.71:0/874518688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad78157700 msgr2=0x7fad781fa370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T15:40:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad78157700 0x7fad781fa370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 -- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad780c4cc0 msgr2=0x7fad781f9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad780c4cc0 0x7fad781f9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 -- 172.21.15.71:0/874518688 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad781e74d0 con 0x7fad780ac670 2024-04-16T15:40:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad777fe640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad780c4cc0 0x7fad781f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:43.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.636+0000 7f501efab640 1 -- start start 2024-04-16T15:40:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7f37363cd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c7ea0 0x7f3730147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7f37363cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37300c1920 con 0x7f3730070000 2024-04-16T15:40:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7f37363cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37301554b0 con 0x7f37300c1f80 2024-04-16T15:40:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7f37363cd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37300c2360 con 0x7f37300c7ea0 2024-04-16T15:40:43.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7fad77fff640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad780ac670 0x7fad780be9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad70037040 tx=0x7fad70004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f2dc45ba640 1 Processor -- start 2024-04-16T15:40:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7feb1affd640 1 -- 172.21.15.71:0/1860712752 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb20005290 con 0x7feb24156f70 2024-04-16T15:40:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f372f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37300c1f80 0x7f3730147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f373496a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c7ea0 0x7f3730147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7feb1affd640 1 -- 172.21.15.71:0/1860712752 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2000a880 con 0x7feb24156f70 2024-04-16T15:40:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f2dc45ba640 1 -- start start 2024-04-16T15:40:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f373496a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c7ea0 0x7f3730147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34328/0 (socket says 172.21.15.71:34328) 2024-04-16T15:40:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f373496a640 1 -- 172.21.15.71:0/4291958729 learned_addr learned my addr 172.21.15.71:0/4291958729 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7fad74ff9640 1 -- 172.21.15.71:0/874518688 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70044020 con 0x7fad780ac670 2024-04-16T15:40:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7fad74ff9640 1 -- 172.21.15.71:0/874518688 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7003f670 con 0x7fad780ac670 2024-04-16T15:40:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7f501efab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100071b0 0x7f5010180980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.637+0000 7f63e1d8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 0x7f63dc15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f63e1d8f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 0x7f63dc15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad780bf080 con 0x7fad780ac670 2024-04-16T15:40:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f63e1d8f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc071df0 con 0x7f63dc06fe40 2024-04-16T15:40:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63dc06fe40 0x7f63dc1f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f63db7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63dc06fe40 0x7f63dc1f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46992/0 (socket says 172.21.15.71:46992) 2024-04-16T15:40:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f63db7fe640 1 -- 172.21.15.71:0/3349130432 learned_addr learned my addr 172.21.15.71:0/3349130432 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc072a00 con 0x7f63dc073200 2024-04-16T15:40:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc157900 con 0x7f63dc0a7ca0 2024-04-16T15:40:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f63daffd640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 0x7f63dc15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7fdddb7fe640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc003be0 0x7fddd40d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f372ffff640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730070000 0x7f37300c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7feb18ff9640 1 -- 172.21.15.71:0/1860712752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb00005290 con 0x7feb24156f70 2024-04-16T15:40:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501efab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501000e9c0 0x7f50100da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501efab640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f501004db50 0x7f501017c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501efab640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f501004adc0 con 0x7f50100071b0 2024-04-16T15:40:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501efab640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f501004aae0 con 0x7f501000e9c0 2024-04-16T15:40:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f373496a640 1 -- 172.21.15.71:0/4291958729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37300c1f80 msgr2=0x7f3730147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f373496a640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37300c1f80 0x7f3730147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f373496a640 1 -- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730070000 msgr2=0x7f37300c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f373496a640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730070000 0x7f37300c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f373496a640 1 -- 172.21.15.71:0/4291958729 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37301ef810 con 0x7f37300c7ea0 2024-04-16T15:40:43.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501efab640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f501004af40 con 0x7f501004db50 2024-04-16T15:40:43.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501e7aa640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f501004db50 0x7f501017c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501d7a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501000e9c0 0x7f50100da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501d7a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501000e9c0 0x7f50100da9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51268/0 (socket says 172.21.15.71:51268) 2024-04-16T15:40:43.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7f501d7a8640 1 -- 172.21.15.71:0/1830054874 learned_addr learned my addr 172.21.15.71:0/1830054874 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f501e7aa640 1 -- 172.21.15.71:0/1830054874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501000e9c0 msgr2=0x7f50100da9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f501e7aa640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501000e9c0 0x7f50100da9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f501e7aa640 1 -- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100071b0 msgr2=0x7f5010180980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f501e7aa640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100071b0 0x7f5010180980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f501e7aa640 1 -- 172.21.15.71:0/1830054874 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50100ddd40 con 0x7f501004db50 2024-04-16T15:40:43.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7fad74ff9640 1 -- 172.21.15.71:0/874518688 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70039040 con 0x7fad780ac670 2024-04-16T15:40:43.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.638+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad780c3510 con 0x7fad780ac670 2024-04-16T15:40:43.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f2dc45ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da800b600 0x7f2dbc14e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f373496a640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c7ea0 0x7f3730147b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f372000d600 tx=0x7f372000dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f2dc45ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc1ea950 0x7f2dbc146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f501e7aa640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f501004db50 0x7f501017c9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f501806a880 tx=0x7f5018075650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f2dc2356640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da800b600 0x7f2dbc14e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f2dc2356640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da800b600 0x7f2dbc14e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34350/0 (socket says 172.21.15.71:34350) 2024-04-16T15:40:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f2dc2356640 1 -- 172.21.15.71:0/1176962353 learned_addr learned my addr 172.21.15.71:0/1176962353 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f2dc1b55640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc1ea950 0x7f2dbc146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7fd3e41a8640 1 -- start start 2024-04-16T15:40:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f63daffd640 1 -- 172.21.15.71:0/3349130432 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 msgr2=0x7f63dc15cd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T15:40:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f63daffd640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 0x7f63dc15cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f63daffd640 1 -- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63dc06fe40 msgr2=0x7f63dc1f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f63daffd640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63dc06fe40 0x7f63dc1f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dbc0ad140 0x7f2dbc0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f63daffd640 1 -- 172.21.15.71:0/3349130432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc0ab850 con 0x7f63dc073200 2024-04-16T15:40:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f63db7fe640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63dc06fe40 0x7f63dc1f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc152a60 con 0x7f2dbc1ea950 2024-04-16T15:40:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.639+0000 7f372cff9640 1 -- 172.21.15.71:0/4291958729 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200125f0 con 0x7f37300c7ea0 2024-04-16T15:40:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f500e7fc640 1 -- 172.21.15.71:0/1830054874 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50180748f0 con 0x7f501004db50 2024-04-16T15:40:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f372cff9640 1 -- 172.21.15.71:0/4291958729 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3720019070 con 0x7f37300c7ea0 2024-04-16T15:40:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f372cff9640 1 -- 172.21.15.71:0/4291958729 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200129d0 con 0x7f37300c7ea0 2024-04-16T15:40:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0c2610 con 0x7f2dbc0ad140 2024-04-16T15:40:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50100dca10 con 0x7f501004db50 2024-04-16T15:40:43.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f2dc2b57640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dbc0ad140 0x7f2dbc0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f37363cd640 1 -- 172.21.15.71:0/4291958729 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37300c6b80 con 0x7f37300c7ea0 2024-04-16T15:40:43.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc14f4c0 con 0x7f2da800b600 2024-04-16T15:40:43.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5010180180 con 0x7f501004db50 2024-04-16T15:40:43.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7f37363cd640 1 -- 172.21.15.71:0/4291958729 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3730147540 con 0x7f37300c7ea0 2024-04-16T15:40:43.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7fdddbfff640 1 -- 172.21.15.71:0/3896869118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddd40428a0 msgr2=0x7fddd416e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7fdddbfff640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddd40428a0 0x7fddd416e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7fdddbfff640 1 -- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc003be0 msgr2=0x7fddd40d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7fdddbfff640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc003be0 0x7fddd40d63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad780c3710 con 0x7fad780ac670 2024-04-16T15:40:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7fdddbfff640 1 -- 172.21.15.71:0/3896869118 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd40d0e80 con 0x7fddd416ecf0 2024-04-16T15:40:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7fd3e41a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3dc0beb50 0x7fd3dc0c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7fad74ff9640 1 -- 172.21.15.71:0/874518688 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad70047020 con 0x7fad780ac670 2024-04-16T15:40:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7fdddbfff640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddd416ecf0 0x7fddd40d1640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdddc065520 tx=0x7fdddc075650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.640+0000 7fddc7fff640 1 -- 172.21.15.71:0/3896869118 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc050300 con 0x7fddd416ecf0 2024-04-16T15:40:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7fddc7fff640 1 -- 172.21.15.71:0/3896869118 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdddc0744e0 con 0x7fddd416ecf0 2024-04-16T15:40:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7fddc7fff640 1 -- 172.21.15.71:0/3896869118 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0748d0 con 0x7fddd416ecf0 2024-04-16T15:40:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd416d130 con 0x7fddd416ecf0 2024-04-16T15:40:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd41700c0 con 0x7fddd416ecf0 2024-04-16T15:40:43.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f63daffd640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 0x7f63dc15c7d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63cc032000 tx=0x7f63cc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f500e7fc640 1 -- 172.21.15.71:0/1830054874 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5018074a90 con 0x7f501004db50 2024-04-16T15:40:43.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f500e7fc640 1 -- 172.21.15.71:0/1830054874 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50180748f0 con 0x7f501004db50 2024-04-16T15:40:43.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f2dc1b55640 1 -- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da800b600 msgr2=0x7f2dbc14e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f2dc1b55640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da800b600 0x7f2dbc14e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f2dc1b55640 1 -- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dbc0ad140 msgr2=0x7f2dbc0c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f2dc1b55640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dbc0ad140 0x7f2dbc0c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f2dc1b55640 1 -- 172.21.15.71:0/1176962353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc0c6eb0 con 0x7f2dbc1ea950 2024-04-16T15:40:43.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f2dc2b57640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dbc0ad140 0x7f2dbc0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:43.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f37363cd640 1 -- 172.21.15.71:0/4291958729 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3730068c10 con 0x7f37300c7ea0 2024-04-16T15:40:43.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7fad74ff9640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fad6406c8f0 0x7fad6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7fad777fe640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fad6406c8f0 0x7fad6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7fad74ff9640 1 -- 172.21.15.71:0/874518688 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fad700ae410 con 0x7fad780ac670 2024-04-16T15:40:43.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7fad777fe640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fad6406c8f0 0x7fad6406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad781f96d0 tx=0x7fad6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd4005390 con 0x7fddd416ecf0 2024-04-16T15:40:43.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7feb1affd640 1 -- 172.21.15.71:0/1860712752 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb2001a070 con 0x7feb24156f70 2024-04-16T15:40:43.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7feb1affd640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feb0806c970 0x7feb0806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7feb1affd640 1 -- 172.21.15.71:0/1860712752 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7feb2008b0d0 con 0x7feb24156f70 2024-04-16T15:40:43.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7feb2a303640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feb0806c970 0x7feb0806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7feb2a303640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feb0806c970 0x7feb0806ee30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb241503a0 tx=0x7feb1c0094f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f63d3fff640 1 -- 172.21.15.71:0/3349130432 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc043030 con 0x7f63dc073200 2024-04-16T15:40:43.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f63d3fff640 1 -- 172.21.15.71:0/3349130432 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc02ce50 con 0x7f63dc073200 2024-04-16T15:40:43.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f63d3fff640 1 -- 172.21.15.71:0/3349130432 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc03e450 con 0x7f63dc073200 2024-04-16T15:40:43.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.641+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc158520 con 0x7f63dc073200 2024-04-16T15:40:43.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc1587c0 con 0x7f63dc073200 2024-04-16T15:40:43.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7fd3e41a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc14c730 0x7fd3dc0bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7fd3e41a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3dc1fb8a0 0x7fd3dc1fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7fd3e41a8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc06e8e0 con 0x7fd3dc0beb50 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7fd3e41a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc0c6630 con 0x7fd3dc1fb8a0 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.642+0000 7fd3e41a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd3dc0c8d60 con 0x7fd3dc14c730 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7fd3e1743640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc14c730 0x7fd3dc0bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7fd3e1743640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc14c730 0x7fd3dc0bfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34366/0 (socket says 172.21.15.71:34366) 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7fd3e1743640 1 -- 172.21.15.71:0/355716087 learned_addr learned my addr 172.21.15.71:0/355716087 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7fd3e1f44640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3dc0beb50 0x7fd3dc0c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7f372cff9640 1 -- 172.21.15.71:0/4291958729 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3720012b70 con 0x7f37300c7ea0 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7f372cff9640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f371c06c940 0x7f371c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7f372cff9640 1 -- 172.21.15.71:0/4291958729 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f372008c520 con 0x7f37300c7ea0 2024-04-16T15:40:43.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7f372ffff640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f371c06c940 0x7f371c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7fd3e1f44640 1 -- 172.21.15.71:0/355716087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc14c730 msgr2=0x7fd3dc0bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:43.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7fd3e1f44640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc14c730 0x7fd3dc0bfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7fd3e1f44640 1 -- 172.21.15.71:0/355716087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3dc1fb8a0 msgr2=0x7fd3dc1fbd50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T15:40:43.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7fd3e1f44640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3dc1fb8a0 0x7fd3dc1fbd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:43.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7f372ffff640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f371c06c940 0x7f371c06ee00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3728035000 tx=0x7f3728030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7fd3e1f44640 1 -- 172.21.15.71:0/355716087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc0c0fa0 con 0x7fd3dc0beb50 2024-04-16T15:40:43.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7fd3e1f44640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3dc0beb50 0x7fd3dc0c3dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c430 tx=0x7fd3d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7f500e7fc640 1 -- 172.21.15.71:0/1830054874 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5018094440 con 0x7f501004db50 2024-04-16T15:40:43.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.643+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50100057e0 con 0x7f501004db50 2024-04-16T15:40:43.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7f2dc1b55640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc1ea950 0x7f2dbc146b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b540 tx=0x7f2db800ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7f63d3fff640 1 -- 172.21.15.71:0/3349130432 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63cc046070 con 0x7f63dc073200 2024-04-16T15:40:43.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7f500e7fc640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4ffc06cb20 0x7f4ffc06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c0005290 con 0x7f63dc073200 2024-04-16T15:40:43.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7f501dfa9640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4ffc06cb20 0x7f4ffc06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7f500e7fc640 1 -- 172.21.15.71:0/1830054874 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f50181072c0 con 0x7f501004db50 2024-04-16T15:40:43.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.644+0000 7f63d3fff640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f63bc06c9c0 0x7f63bc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.645+0000 7f63db7fe640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f63bc06c9c0 0x7f63bc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.645+0000 7f501dfa9640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4ffc06cb20 0x7f4ffc06efe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5014006b10 tx=0x7f5014008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.645+0000 7f63d3fff640 1 -- 172.21.15.71:0/3349130432 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f63cc0adeb0 con 0x7f63dc073200 2024-04-16T15:40:43.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.645+0000 7f63db7fe640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f63bc06c9c0 0x7f63bc06ee80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63c40011e0 tx=0x7f63c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.646+0000 7fad74ff9640 1 -- 172.21.15.71:0/874518688 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad780c3710 con 0x7fad780ac670 2024-04-16T15:40:43.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7fddc7fff640 1 -- 172.21.15.71:0/3896869118 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdddc0943c0 con 0x7fddd416ecf0 2024-04-16T15:40:43.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7fddc7fff640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fddc006c8f0 0x7fddc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7feb1affd640 1 -- 172.21.15.71:0/1860712752 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feb241f0950 con 0x7feb24156f70 2024-04-16T15:40:43.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7fd3d27fc640 1 -- 172.21.15.71:0/355716087 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8017070 con 0x7fd3dc0beb50 2024-04-16T15:40:43.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7fd3d27fc640 1 -- 172.21.15.71:0/355716087 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d8005ce0 con 0x7fd3dc0beb50 2024-04-16T15:40:43.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7fd3d27fc640 1 -- 172.21.15.71:0/355716087 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80072d0 con 0x7fd3dc0beb50 2024-04-16T15:40:43.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc0c1ae0 con 0x7fd3dc0beb50 2024-04-16T15:40:43.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc0c03a0 con 0x7fd3dc0beb50 2024-04-16T15:40:43.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7fdddb7fe640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fddc006c8f0 0x7fddc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7f2db2ffd640 1 -- 172.21.15.71:0/1176962353 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8018070 con 0x7f2dbc1ea950 2024-04-16T15:40:43.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7f2db2ffd640 1 -- 172.21.15.71:0/1176962353 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db8014700 con 0x7f2dbc1ea950 2024-04-16T15:40:43.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.648+0000 7f2db2ffd640 1 -- 172.21.15.71:0/1176962353 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8014ad0 con 0x7f2dbc1ea950 2024-04-16T15:40:43.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.649+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc152050 con 0x7f2dbc1ea950 2024-04-16T15:40:43.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.649+0000 7fdddb7fe640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fddc006c8f0 0x7fddc006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fddd0034000 tx=0x7fddd0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.649+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc150bc0 con 0x7f2dbc1ea950 2024-04-16T15:40:43.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.650+0000 7f63d3fff640 1 -- 172.21.15.71:0/3349130432 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63cc04d070 con 0x7f63dc073200 2024-04-16T15:40:43.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.650+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc14b370 con 0x7fd3dc0beb50 2024-04-16T15:40:43.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.651+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dbc0a7ca0 con 0x7f2dbc1ea950 2024-04-16T15:40:43.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.651+0000 7fd3d27fc640 1 -- 172.21.15.71:0/355716087 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3d801a070 con 0x7fd3dc0beb50 2024-04-16T15:40:43.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.651+0000 7f2db2ffd640 1 -- 172.21.15.71:0/1176962353 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2db801b070 con 0x7f2dbc1ea950 2024-04-16T15:40:43.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.652+0000 7f372cff9640 1 -- 172.21.15.71:0/4291958729 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f372005a5f0 con 0x7f37300c7ea0 2024-04-16T15:40:43.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.651+0000 7fd3d27fc640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd3c006c8f0 0x7fd3c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.651+0000 7fd3d27fc640 1 -- 172.21.15.71:0/355716087 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd3d808b020 con 0x7fd3dc0beb50 2024-04-16T15:40:43.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.652+0000 7fd3e1743640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd3c006c8f0 0x7fd3c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.652+0000 7f2db2ffd640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2da006c920 0x7f2da006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:43.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.652+0000 7f2db2ffd640 1 -- 172.21.15.71:0/1176962353 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2db808c040 con 0x7f2dbc1ea950 2024-04-16T15:40:43.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.652+0000 7fd3e1743640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd3c006c8f0 0x7fd3c006edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc02b910 tx=0x7fd3cc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.652+0000 7f2dc2356640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2da006c920 0x7f2da006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:43.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.653+0000 7f2dc2356640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2da006c920 0x7f2da006ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2db4001b60 tx=0x7f2db40019d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:43.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.656+0000 7fddc7fff640 1 -- 172.21.15.71:0/3896869118 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdddc096e10 con 0x7fddd416ecf0 2024-04-16T15:40:43.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.656+0000 7f500e7fc640 1 -- 172.21.15.71:0/1830054874 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50180d53a0 con 0x7f501004db50 2024-04-16T15:40:43.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.660+0000 7fddc7fff640 1 -- 172.21.15.71:0/3896869118 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdddc0d5450 con 0x7fddd416ecf0 2024-04-16T15:40:43.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.661+0000 7fd3d27fc640 1 -- 172.21.15.71:0/355716087 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3dc14b370 con 0x7fd3dc0beb50 2024-04-16T15:40:43.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:43.670+0000 7f2db2ffd640 1 -- 172.21.15.71:0/1176962353 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2dbc0a7ca0 con 0x7f2dbc1ea950 2024-04-16T15:40:44.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.065+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f63c0005580 con 0x7f63dc073200 2024-04-16T15:40:44.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.066+0000 7f63d3fff640 1 -- 172.21.15.71:0/3349130432 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f63cc0333a0 con 0x7f63dc073200 2024-04-16T15:40:44.065 INFO:teuthology.orchestra.run.smithi071.stdout:55834574858 2024-04-16T15:40:44.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.067+0000 7f37363cd640 1 -- 172.21.15.71:0/4291958729 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f373006fde0 con 0x7f37300c7ea0 2024-04-16T15:40:44.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.068+0000 7f372cff9640 1 -- 172.21.15.71:0/4291958729 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f372005a5f0 con 0x7f37300c7ea0 2024-04-16T15:40:44.067 INFO:teuthology.orchestra.run.smithi071.stdout:55834574857 2024-04-16T15:40:44.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f63bc06c9c0 msgr2=0x7f63bc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7feb18ff9640 1 -- 172.21.15.71:0/1860712752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feb00005580 con 0x7feb24156f70 2024-04-16T15:40:44.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f63bc06c9c0 0x7f63bc06ee80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63c40011e0 tx=0x7f63c4009040 comp rx=0 tx=0).stop 2024-04-16T15:40:44.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 msgr2=0x7f63dc15c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 0x7f63dc15c7d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63cc032000 tx=0x7f63cc004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:44.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 shutdown_connections 2024-04-16T15:40:44.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f63bc06c9c0 0x7f63bc06ee80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7feb1affd640 1 -- 172.21.15.71:0/1860712752 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb200010c0 con 0x7feb24156f70 2024-04-16T15:40:44.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f63dc0a7ca0 0x7f63dc15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63dc073200 0x7f63dc15c7d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.069+0000 7f63e1d8f640 1 --2- 172.21.15.71:0/3349130432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63dc06fe40 0x7f63dc1f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.070+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 >> 172.21.15.71:0/3349130432 conn(0x7f63dc06b930 msgr2=0x7f63dc154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.071 INFO:teuthology.orchestra.run.smithi071.stdout:55834574859 2024-04-16T15:40:44.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.070+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 shutdown_connections 2024-04-16T15:40:44.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.070+0000 7f63e1d8f640 1 -- 172.21.15.71:0/3349130432 wait complete. 2024-04-16T15:40:44.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 -- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f371c06c940 msgr2=0x7f371c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f371c06c940 0x7f371c06ee00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3728035000 tx=0x7f3728030000 comp rx=0 tx=0).stop 2024-04-16T15:40:44.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 -- 172.21.15.71:0/4291958729 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c7ea0 msgr2=0x7f3730147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c7ea0 0x7f3730147b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f372000d600 tx=0x7f372000dac0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 -- 172.21.15.71:0/4291958729 shutdown_connections 2024-04-16T15:40:44.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f371c06c940 0x7f371c06ee00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37300c7ea0 0x7f3730147b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37300c1f80 0x7f3730147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 --2- 172.21.15.71:0/4291958729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730070000 0x7f37300c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.071+0000 7f37167fc640 1 -- 172.21.15.71:0/4291958729 >> 172.21.15.71:0/4291958729 conn(0x7f37300c7310 msgr2=0x7f37300c3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.072+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fad78068be0 con 0x7fad780ac670 2024-04-16T15:40:44.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.072+0000 7f37167fc640 1 -- 172.21.15.71:0/4291958729 shutdown_connections 2024-04-16T15:40:44.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 -- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feb0806c970 msgr2=0x7feb0806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feb0806c970 0x7feb0806ee30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb241503a0 tx=0x7feb1c0094f0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 -- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24156f70 msgr2=0x7feb241efed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24156f70 0x7feb241efed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c780 tx=0x7feb2000cc40 comp rx=0 tx=0).stop 2024-04-16T15:40:44.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7f37167fc640 1 -- 172.21.15.71:0/4291958729 wait complete. 2024-04-16T15:40:44.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 -- 172.21.15.71:0/1860712752 shutdown_connections 2024-04-16T15:40:44.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7feb0806c970 0x7feb0806ee30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb24157be0 0x7feb241f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24156f70 0x7feb241efed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 --2- 172.21.15.71:0/1860712752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb240c62d0 0x7feb24151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 -- 172.21.15.71:0/1860712752 >> 172.21.15.71:0/1860712752 conn(0x7feb240ab440 msgr2=0x7feb240c2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7fad74ff9640 1 -- 172.21.15.71:0/874518688 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fad78072550 con 0x7fad780ac670 2024-04-16T15:40:44.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.073+0000 7feb18ff9640 1 -- 172.21.15.71:0/1860712752 shutdown_connections 2024-04-16T15:40:44.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.074+0000 7feb18ff9640 1 -- 172.21.15.71:0/1860712752 wait complete. 2024-04-16T15:40:44.077 INFO:teuthology.orchestra.run.smithi071.stdout:55834574858 2024-04-16T15:40:44.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.074+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5010042af0 con 0x7f501004db50 2024-04-16T15:40:44.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.074+0000 7f500e7fc640 1 -- 172.21.15.71:0/1830054874 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f50180d4ca0 con 0x7f501004db50 2024-04-16T15:40:44.077 INFO:teuthology.orchestra.run.smithi071.stdout:55834574857 2024-04-16T15:40:44.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.077+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fad6406c8f0 msgr2=0x7fad6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.077+0000 7fad7e759640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fad6406c8f0 0x7fad6406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad781f96d0 tx=0x7fad6c009040 comp rx=0 tx=0).stop 2024-04-16T15:40:44.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.077+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad780ac670 msgr2=0x7fad780be9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.077+0000 7fad7e759640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad780ac670 0x7fad780be9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad70037040 tx=0x7fad70004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:44.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.078+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4ffc06cb20 msgr2=0x7f4ffc06efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.078+0000 7f501efab640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4ffc06cb20 0x7f4ffc06efe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5014006b10 tx=0x7f5014008040 comp rx=0 tx=0).stop 2024-04-16T15:40:44.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.078+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f501004db50 msgr2=0x7f501017c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.078+0000 7f501efab640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f501004db50 0x7f501017c9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f501806a880 tx=0x7f5018075650 comp rx=0 tx=0).stop 2024-04-16T15:40:44.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 shutdown_connections 2024-04-16T15:40:44.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7f501efab640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4ffc06cb20 0x7f4ffc06efe0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7f501efab640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f501004db50 0x7f501017c9c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7f501efab640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f501000e9c0 0x7f50100da9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7f501efab640 1 --2- 172.21.15.71:0/1830054874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100071b0 0x7f5010180980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 >> 172.21.15.71:0/1830054874 conn(0x7f501004c660 msgr2=0x7f501004ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 shutdown_connections 2024-04-16T15:40:44.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 shutdown_connections 2024-04-16T15:40:44.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7fad7e759640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fad6406c8f0 0x7fad6406edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7fad7e759640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad78157700 0x7fad781fa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7fad7e759640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad780c4cc0 0x7fad781f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7fad7e759640 1 --2- 172.21.15.71:0/874518688 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad780ac670 0x7fad780be9f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 >> 172.21.15.71:0/874518688 conn(0x7fad780ab270 msgr2=0x7fad78158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7f501efab640 1 -- 172.21.15.71:0/1830054874 wait complete. 2024-04-16T15:40:44.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 shutdown_connections 2024-04-16T15:40:44.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.079+0000 7fad7e759640 1 -- 172.21.15.71:0/874518688 wait complete. 2024-04-16T15:40:44.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.082+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fddd400b590 con 0x7fddd416ecf0 2024-04-16T15:40:44.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.082+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd3dc068be0 con 0x7fd3dc0beb50 2024-04-16T15:40:44.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.083+0000 7fd3d27fc640 1 -- 172.21.15.71:0/355716087 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd3d8058b50 con 0x7fd3dc0beb50 2024-04-16T15:40:44.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.083+0000 7fddc7fff640 1 -- 172.21.15.71:0/3896869118 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdddc0d4d50 con 0x7fddd416ecf0 2024-04-16T15:40:44.086 INFO:teuthology.orchestra.run.smithi071.stdout:55834574859 2024-04-16T15:40:44.087 INFO:teuthology.orchestra.run.smithi071.stdout:55834574859 2024-04-16T15:40:44.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.085+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2dbc068be0 con 0x7f2dbc1ea950 2024-04-16T15:40:44.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.086+0000 7f2db2ffd640 1 -- 172.21.15.71:0/1176962353 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2db8059ad0 con 0x7f2dbc1ea950 2024-04-16T15:40:44.087 INFO:teuthology.orchestra.run.smithi071.stdout:55834574858 2024-04-16T15:40:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fddc006c8f0 msgr2=0x7fddc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd3c006c8f0 msgr2=0x7fd3c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fdde0e77640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fddc006c8f0 0x7fddc006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fddd0034000 tx=0x7fddd0009030 comp rx=0 tx=0).stop 2024-04-16T15:40:44.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd3c006c8f0 0x7fd3c006edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc02b910 tx=0x7fd3cc008030 comp rx=0 tx=0).stop 2024-04-16T15:40:44.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddd416ecf0 msgr2=0x7fddd40d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3dc0beb50 msgr2=0x7fd3dc0c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3dc0beb50 0x7fd3dc0c3dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c430 tx=0x7fd3d800c8f0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fdde0e77640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddd416ecf0 0x7fddd40d1640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdddc065520 tx=0x7fdddc075650 comp rx=0 tx=0).stop 2024-04-16T15:40:44.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 shutdown_connections 2024-04-16T15:40:44.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fdde0e77640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fddc006c8f0 0x7fddc006edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fdde0e77640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fddd416ecf0 0x7fddd40d1640 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.087+0000 7fdde0e77640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddd40428a0 0x7fddd416e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fdde0e77640 1 --2- 172.21.15.71:0/3896869118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddcc003be0 0x7fddd40d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 shutdown_connections 2024-04-16T15:40:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd3c006c8f0 0x7fd3c006edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3dc1fb8a0 0x7fd3dc1fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd3dc14c730 0x7fd3dc0bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fd3e41a8640 1 --2- 172.21.15.71:0/355716087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3dc0beb50 0x7fd3dc0c3dc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 >> 172.21.15.71:0/355716087 conn(0x7fd3dc0ad5e0 msgr2=0x7fd3dc0ac0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 >> 172.21.15.71:0/3896869118 conn(0x7fddd4048250 msgr2=0x7fddd40e2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 shutdown_connections 2024-04-16T15:40:44.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fdde0e77640 1 -- 172.21.15.71:0/3896869118 wait complete. 2024-04-16T15:40:44.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 shutdown_connections 2024-04-16T15:40:44.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.088+0000 7fd3e41a8640 1 -- 172.21.15.71:0/355716087 wait complete. 2024-04-16T15:40:44.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2da006c920 msgr2=0x7f2da006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2da006c920 0x7f2da006ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2db4001b60 tx=0x7f2db40019d0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc1ea950 msgr2=0x7f2dbc146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc1ea950 0x7f2dbc146b60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b540 tx=0x7f2db800ba00 comp rx=0 tx=0).stop 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 shutdown_connections 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2da006c920 0x7f2da006ede0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dbc0ad140 0x7f2dbc0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc1ea950 0x7f2dbc146b60 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b540 tx=0x7f2db800ba00 comp rx=0 tx=0).stop 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 --2- 172.21.15.71:0/1176962353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2da800b600 0x7f2dbc14e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 >> 172.21.15.71:0/1176962353 conn(0x7f2dbc06b930 msgr2=0x7f2dbc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 shutdown_connections 2024-04-16T15:40:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.090+0000 7f2dc45ba640 1 -- 172.21.15.71:0/1176962353 wait complete. 2024-04-16T15:40:44.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-04-16T15:40:44.097 DEBUG:teuthology.parallel:result is None 2024-04-16T15:40:44.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2024-04-16T15:40:44.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-04-16T15:40:44.098 DEBUG:teuthology.parallel:result is None 2024-04-16T15:40:44.098 DEBUG:teuthology.parallel:result is None 2024-04-16T15:40:44.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-04-16T15:40:44.101 DEBUG:teuthology.parallel:result is None 2024-04-16T15:40:44.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-04-16T15:40:44.102 DEBUG:teuthology.parallel:result is None 2024-04-16T15:40:44.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-16T15:40:44.106 DEBUG:teuthology.parallel:result is None 2024-04-16T15:40:44.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-04-16T15:40:44.106 DEBUG:teuthology.parallel:result is None 2024-04-16T15:40:44.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2024-04-16T15:40:44.108 DEBUG:teuthology.parallel:result is None 2024-04-16T15:40:44.108 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-16T15:40:44.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:40:44.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.250+0000 7f42fb57c640 1 Processor -- start 2024-04-16T15:40:44.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.250+0000 7f42fb57c640 1 -- start start 2024-04-16T15:40:44.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42fb57c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f4153570 0x7f42f40c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42fb57c640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f40c82d0 con 0x7f42f40c4350 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42fb57c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f4155d70 con 0x7f42f4159740 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42fb57c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f42f415a2e0 con 0x7f42f4153570 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42f8b17640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f42f40c4350 0x7f42f40c5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34262/0 (socket says 172.21.15.71:34262) 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42f8b17640 1 -- 172.21.15.71:0/1159119611 learned_addr learned my addr 172.21.15.71:0/1159119611 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413141070 0 0) 0x7f42f40c82d0 con 0x7f42f40c4350 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42f9318640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f4153570 0x7f42f40c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.251+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42e0003520 con 0x7f42f40c4350 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1962241965 0 0) 0x7f42e0003520 con 0x7f42f40c4350 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f4153570 msgr2=0x7f42f40c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f4153570 0x7f42f40c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 >> v1:172.21.15.122:6789/0 conn(0x7f42f4159740 legacy=0x7f42f4159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f40c8d50 con 0x7f42f40c4350 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f42e4001670 con 0x7f42f40c4350 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 >> v1:172.21.15.71:6789/0 conn(0x7f42f40c4350 legacy=0x7f42f40c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f4153570 0x7f42e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0003f20 0x7f42e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f42f415a2e0 con 0x7f42e0004d40 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42e000bb80 con 0x7f42e0003f20 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f42e000bd80 con 0x7f42f4153570 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42f9b19640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42f9318640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f4153570 0x7f42e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42f8b17640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0003f20 0x7f42e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42f9b19640 1 -- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f4153570 msgr2=0x7f42e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42f9b19640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f4153570 0x7f42e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42f9b19640 1 -- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0003f20 msgr2=0x7f42e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42f9b19640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42e0003f20 0x7f42e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.252+0000 7f42f9b19640 1 -- 172.21.15.71:0/1159119611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e0003520 con 0x7f42e0004d40 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.253+0000 7f42f9b19640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42e00051c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42ec001860 tx=0x7f42e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ad87e2d8b5f0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.253+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec038070 con 0x7f42e0004d40 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.253+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ec033410 con 0x7f42e0004d40 2024-04-16T15:40:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.253+0000 7f42ebfff640 1 -- 172.21.15.71:0/1159119611 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec001430 con 0x7f42e0004d40 2024-04-16T15:40:44.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.253+0000 7f42fb57c640 1 -- 172.21.15.71:0/1159119611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 msgr2=0x7f42e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.253+0000 7f42fb57c640 1 --2- 172.21.15.71:0/1159119611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42e00051c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42ec001860 tx=0x7f42e0005830 comp rx=0 tx=0).stop 2024-04-16T15:40:44.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.253+0000 7f42f9b19640 1 -- 172.21.15.71:0/1159119611 reap_dead start 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.254+0000 7f42fb57c640 1 -- 172.21.15.71:0/1159119611 shutdown_connections 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.254+0000 7f42fb57c640 1 -- 172.21.15.71:0/1159119611 >> 172.21.15.71:0/1159119611 conn(0x7f42f40c96e0 msgr2=0x7f42f40c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.254+0000 7f42fb57c640 1 -- 172.21.15.71:0/1159119611 shutdown_connections 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.254+0000 7f42fb57c640 1 -- 172.21.15.71:0/1159119611 wait complete. 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42fb57c640 1 Processor -- start 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42fb57c640 1 -- start start 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42fb57c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42f406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42fb57c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42f4070800 0x7f42f40aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42fb57c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f40ab500 0x7f42f40a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42fb57c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40c8ff0 con 0x7f42e0004d40 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42fb57c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40c64b0 con 0x7f42f4070800 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42fb57c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f42f40c6630 con 0x7f42f40ab500 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42f9318640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42f406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42f9318640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42f406e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47062/0 (socket says 172.21.15.71:47062) 2024-04-16T15:40:44.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.255+0000 7f42f9318640 1 -- 172.21.15.71:0/3432977353 learned_addr learned my addr 172.21.15.71:0/3432977353 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f8b17640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42f4070800 0x7f42f40aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f9b19640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f40ab500 0x7f42f40a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f9318640 1 -- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f40ab500 msgr2=0x7f42f40a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f9318640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f40ab500 0x7f42f40a9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f9318640 1 -- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42f4070800 msgr2=0x7f42f40aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f9318640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42f4070800 0x7f42f40aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f9318640 1 -- 172.21.15.71:0/3432977353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f40712b0 con 0x7f42e0004d40 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f8b17640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42f4070800 0x7f42f40aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42f9318640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42f406e3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f42f0011e90 tx=0x7f42f000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42e9ffb640 1 -- 172.21.15.71:0/3432977353 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0015630 con 0x7f42e0004d40 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42e9ffb640 1 -- 172.21.15.71:0/3432977353 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f00157d0 con 0x7f42e0004d40 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f4071590 con 0x7f42e0004d40 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42e9ffb640 1 -- 172.21.15.71:0/3432977353 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0015ba0 con 0x7f42e0004d40 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.256+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f40a95a0 con 0x7f42e0004d40 2024-04-16T15:40:44.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.257+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f4068d20 con 0x7f42e0004d40 2024-04-16T15:40:44.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.258+0000 7f42e9ffb640 1 -- 172.21.15.71:0/3432977353 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42f001d400 con 0x7f42e0004d40 2024-04-16T15:40:44.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.258+0000 7f42e9ffb640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f42d806c8f0 0x7f42d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.258+0000 7f42e9ffb640 1 -- 172.21.15.71:0/3432977353 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f42f008d080 con 0x7f42e0004d40 2024-04-16T15:40:44.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.258+0000 7f42f8b17640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f42d806c8f0 0x7f42d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.259+0000 7f42f8b17640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f42d806c8f0 0x7f42d806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f42f41e9230 tx=0x7f42e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.263+0000 7f42e9ffb640 1 -- 172.21.15.71:0/3432977353 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42f0019030 con 0x7f42e0004d40 2024-04-16T15:40:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.435+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42f40c7c50 con 0x7f42d806c8f0 2024-04-16T15:40:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.442+0000 7f42e9ffb640 1 -- 172.21.15.71:0/3432977353 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f42f40c7c50 con 0x7f42d806c8f0 2024-04-16T15:40:44.446 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:44.448 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f42d806c8f0 msgr2=0x7f42d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f42d806c8f0 0x7f42d806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f42f41e9230 tx=0x7f42e4009040 comp rx=0 tx=0).stop 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 msgr2=0x7f42f406e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42f406e3a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f42f0011e90 tx=0x7f42f000f580 comp rx=0 tx=0).stop 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 shutdown_connections 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f42d806c8f0 0x7f42d806edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f42f40ab500 0x7f42f40a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42f4070800 0x7f42f40aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 --2- 172.21.15.71:0/3432977353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42e0004d40 0x7f42f406e3a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 >> 172.21.15.71:0/3432977353 conn(0x7f42f40c96e0 msgr2=0x7f42f40c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 shutdown_connections 2024-04-16T15:40:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.446+0000 7f42fb57c640 1 -- 172.21.15.71:0/3432977353 wait complete. 2024-04-16T15:40:44.457 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-16T15:40:43.699345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219396,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755324,"statfs":{"total":773094113280,"available":772869451776,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002569"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919216+0000","last_change":"2024-04-16T15:40:23.532452+0000","last_active":"2024-04-16T15:40:29.919216+0000","last_peered":"2024-04-16T15:40:29.919216+0000","last_clean":"2024-04-16T15:40:29.919216+0000","last_became_active":"2024-04-16T15:40:23.531709+0000","last_became_peered":"2024-04-16T15:40:23.531709+0000","last_unstale":"2024-04-16T15:40:29.919216+0000","last_undegraded":"2024-04-16T15:40:29.919216+0000","last_fullsized":"2024-04-16T15:40:29.919216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:52.586318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567735+0000","last_change":"2024-04-16T15:40:24.542948+0000","last_active":"2024-04-16T15:40:29.567735+0000","last_peered":"2024-04-16T15:40:29.567735+0000","last_clean":"2024-04-16T15:40:29.567735+0000","last_became_active":"2024-04-16T15:40:24.537898+0000","last_became_peered":"2024-04-16T15:40:24.537898+0000","last_unstale":"2024-04-16T15:40:29.567735+0000","last_undegraded":"2024-04-16T15:40:29.567735+0000","last_fullsized":"2024-04-16T15:40:29.567735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:18.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566318+0000","last_change":"2024-04-16T15:40:28.807073+0000","last_active":"2024-04-16T15:40:29.566318+0000","last_peered":"2024-04-16T15:40:29.566318+0000","last_clean":"2024-04-16T15:40:29.566318+0000","last_became_active":"2024-04-16T15:40:28.806446+0000","last_became_peered":"2024-04-16T15:40:28.806446+0000","last_unstale":"2024-04-16T15:40:29.566318+0000","last_undegraded":"2024-04-16T15:40:29.566318+0000","last_fullsized":"2024-04-16T15:40:29.566318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:54.213728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567940+0000","last_change":"2024-04-16T15:40:23.526932+0000","last_active":"2024-04-16T15:40:29.567940+0000","last_peered":"2024-04-16T15:40:29.567940+0000","last_clean":"2024-04-16T15:40:29.567940+0000","last_became_active":"2024-04-16T15:40:23.526663+0000","last_became_peered":"2024-04-16T15:40:23.526663+0000","last_unstale":"2024-04-16T15:40:29.567940+0000","last_undegraded":"2024-04-16T15:40:29.567940+0000","last_fullsized":"2024-04-16T15:40:29.567940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:52.586318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919103+0000","last_change":"2024-04-16T15:40:24.541524+0000","last_active":"2024-04-16T15:40:29.919103+0000","last_peered":"2024-04-16T15:40:29.919103+0000","last_clean":"2024-04-16T15:40:29.919103+0000","last_became_active":"2024-04-16T15:40:24.540243+0000","last_became_peered":"2024-04-16T15:40:24.540243+0000","last_unstale":"2024-04-16T15:40:29.919103+0000","last_undegraded":"2024-04-16T15:40:29.919103+0000","last_fullsized":"2024-04-16T15:40:29.919103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919188+0000","last_change":"2024-04-16T15:40:28.569991+0000","last_active":"2024-04-16T15:40:29.919188+0000","last_peered":"2024-04-16T15:40:29.919188+0000","last_clean":"2024-04-16T15:40:29.919188+0000","last_became_active":"2024-04-16T15:40:28.569596+0000","last_became_peered":"2024-04-16T15:40:28.569596+0000","last_unstale":"2024-04-16T15:40:29.919188+0000","last_undegraded":"2024-04-16T15:40:29.919188+0000","last_fullsized":"2024-04-16T15:40:29.919188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:21.999059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921685+0000","last_change":"2024-04-16T15:40:23.535294+0000","last_active":"2024-04-16T15:40:29.921685+0000","last_peered":"2024-04-16T15:40:29.921685+0000","last_clean":"2024-04-16T15:40:29.921685+0000","last_became_active":"2024-04-16T15:40:23.534958+0000","last_became_peered":"2024-04-16T15:40:23.534958+0000","last_unstale":"2024-04-16T15:40:29.921685+0000","last_undegraded":"2024-04-16T15:40:29.921685+0000","last_fullsized":"2024-04-16T15:40:29.921685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:59.914744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919928+0000","last_change":"2024-04-16T15:40:24.544556+0000","last_active":"2024-04-16T15:40:29.919928+0000","last_peered":"2024-04-16T15:40:29.919928+0000","last_clean":"2024-04-16T15:40:29.919928+0000","last_became_active":"2024-04-16T15:40:24.544072+0000","last_became_peered":"2024-04-16T15:40:24.544072+0000","last_unstale":"2024-04-16T15:40:29.919928+0000","last_undegraded":"2024-04-16T15:40:29.919928+0000","last_fullsized":"2024-04-16T15:40:29.919928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:25.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920842+0000","last_change":"2024-04-16T15:40:28.570724+0000","last_active":"2024-04-16T15:40:29.920842+0000","last_peered":"2024-04-16T15:40:29.920842+0000","last_clean":"2024-04-16T15:40:29.920842+0000","last_became_active":"2024-04-16T15:40:28.570338+0000","last_became_peered":"2024-04-16T15:40:28.570338+0000","last_unstale":"2024-04-16T15:40:29.920842+0000","last_undegraded":"2024-04-16T15:40:29.920842+0000","last_fullsized":"2024-04-16T15:40:29.920842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:55.765984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920279+0000","last_change":"2024-04-16T15:40:23.536319+0000","last_active":"2024-04-16T15:40:29.920279+0000","last_peered":"2024-04-16T15:40:29.920279+0000","last_clean":"2024-04-16T15:40:29.920279+0000","last_became_active":"2024-04-16T15:40:23.536059+0000","last_became_peered":"2024-04-16T15:40:23.536059+0000","last_unstale":"2024-04-16T15:40:29.920279+0000","last_undegraded":"2024-04-16T15:40:29.920279+0000","last_fullsized":"2024-04-16T15:40:29.920279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:19.974713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920444+0000","last_change":"2024-04-16T15:40:24.543661+0000","last_active":"2024-04-16T15:40:29.920444+0000","last_peered":"2024-04-16T15:40:29.920444+0000","last_clean":"2024-04-16T15:40:29.920444+0000","last_became_active":"2024-04-16T15:40:24.543414+0000","last_became_peered":"2024-04-16T15:40:24.543414+0000","last_unstale":"2024-04-16T15:40:29.920444+0000","last_undegraded":"2024-04-16T15:40:29.920444+0000","last_fullsized":"2024-04-16T15:40:29.920444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:38.664081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919473+0000","last_change":"2024-04-16T15:40:28.566254+0000","last_active":"2024-04-16T15:40:29.919473+0000","last_peered":"2024-04-16T15:40:29.919473+0000","last_clean":"2024-04-16T15:40:29.919473+0000","last_became_active":"2024-04-16T15:40:28.565804+0000","last_became_peered":"2024-04-16T15:40:28.565804+0000","last_unstale":"2024-04-16T15:40:29.919473+0000","last_undegraded":"2024-04-16T15:40:29.919473+0000","last_fullsized":"2024-04-16T15:40:29.919473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:40.008247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566632+0000","last_change":"2024-04-16T15:40:24.539072+0000","last_active":"2024-04-16T15:40:29.566632+0000","last_peered":"2024-04-16T15:40:29.566632+0000","last_clean":"2024-04-16T15:40:29.566632+0000","last_became_active":"2024-04-16T15:40:24.537100+0000","last_became_peered":"2024-04-16T15:40:24.537100+0000","last_unstale":"2024-04-16T15:40:29.566632+0000","last_undegraded":"2024-04-16T15:40:29.566632+0000","last_fullsized":"2024-04-16T15:40:29.566632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920246+0000","last_change":"2024-04-16T15:40:23.537232+0000","last_active":"2024-04-16T15:40:29.920246+0000","last_peered":"2024-04-16T15:40:29.920246+0000","last_clean":"2024-04-16T15:40:29.920246+0000","last_became_active":"2024-04-16T15:40:23.536233+0000","last_became_peered":"2024-04-16T15:40:23.536233+0000","last_unstale":"2024-04-16T15:40:29.920246+0000","last_undegraded":"2024-04-16T15:40:29.920246+0000","last_fullsized":"2024-04-16T15:40:29.920246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:43.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921321+0000","last_change":"2024-04-16T15:40:28.570768+0000","last_active":"2024-04-16T15:40:29.921321+0000","last_peered":"2024-04-16T15:40:29.921321+0000","last_clean":"2024-04-16T15:40:29.921321+0000","last_became_active":"2024-04-16T15:40:28.570416+0000","last_became_peered":"2024-04-16T15:40:28.570416+0000","last_unstale":"2024-04-16T15:40:29.921321+0000","last_undegraded":"2024-04-16T15:40:29.921321+0000","last_fullsized":"2024-04-16T15:40:29.921321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567237+0000","last_change":"2024-04-16T15:40:24.548809+0000","last_active":"2024-04-16T15:40:29.567237+0000","last_peered":"2024-04-16T15:40:29.567237+0000","last_clean":"2024-04-16T15:40:29.567237+0000","last_became_active":"2024-04-16T15:40:24.548463+0000","last_became_peered":"2024-04-16T15:40:24.548463+0000","last_unstale":"2024-04-16T15:40:29.567237+0000","last_undegraded":"2024-04-16T15:40:29.567237+0000","last_fullsized":"2024-04-16T15:40:29.567237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769455+0000","last_change":"2024-04-16T15:40:23.531263+0000","last_active":"2024-04-16T15:40:29.769455+0000","last_peered":"2024-04-16T15:40:29.769455+0000","last_clean":"2024-04-16T15:40:29.769455+0000","last_became_active":"2024-04-16T15:40:23.530998+0000","last_became_peered":"2024-04-16T15:40:23.530998+0000","last_unstale":"2024-04-16T15:40:29.769455+0000","last_undegraded":"2024-04-16T15:40:29.769455+0000","last_fullsized":"2024-04-16T15:40:29.769455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919978+0000","last_change":"2024-04-16T15:40:28.569962+0000","last_active":"2024-04-16T15:40:29.919978+0000","last_peered":"2024-04-16T15:40:29.919978+0000","last_clean":"2024-04-16T15:40:29.919978+0000","last_became_active":"2024-04-16T15:40:28.569572+0000","last_became_peered":"2024-04-16T15:40:28.569572+0000","last_unstale":"2024-04-16T15:40:29.919978+0000","last_undegraded":"2024-04-16T15:40:29.919978+0000","last_fullsized":"2024-04-16T15:40:29.919978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:18.103136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566305+0000","last_change":"2024-04-16T15:40:24.544276+0000","last_active":"2024-04-16T15:40:29.566305+0000","last_peered":"2024-04-16T15:40:29.566305+0000","last_clean":"2024-04-16T15:40:29.566305+0000","last_became_active":"2024-04-16T15:40:24.544035+0000","last_became_peered":"2024-04-16T15:40:24.544035+0000","last_unstale":"2024-04-16T15:40:29.566305+0000","last_undegraded":"2024-04-16T15:40:29.566305+0000","last_fullsized":"2024-04-16T15:40:29.566305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769209+0000","last_change":"2024-04-16T15:40:23.530210+0000","last_active":"2024-04-16T15:40:29.769209+0000","last_peered":"2024-04-16T15:40:29.769209+0000","last_clean":"2024-04-16T15:40:29.769209+0000","last_became_active":"2024-04-16T15:40:23.529955+0000","last_became_peered":"2024-04-16T15:40:23.529955+0000","last_unstale":"2024-04-16T15:40:29.769209+0000","last_undegraded":"2024-04-16T15:40:29.769209+0000","last_fullsized":"2024-04-16T15:40:29.769209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:21.943427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769997+0000","last_change":"2024-04-16T15:40:28.805304+0000","last_active":"2024-04-16T15:40:29.769997+0000","last_peered":"2024-04-16T15:40:29.769997+0000","last_clean":"2024-04-16T15:40:29.769997+0000","last_became_active":"2024-04-16T15:40:28.804926+0000","last_became_peered":"2024-04-16T15:40:28.804926+0000","last_unstale":"2024-04-16T15:40:29.769997+0000","last_undegraded":"2024-04-16T15:40:29.769997+0000","last_fullsized":"2024-04-16T15:40:29.769997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:25.828695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919801+0000","last_change":"2024-04-16T15:40:24.537825+0000","last_active":"2024-04-16T15:40:29.919801+0000","last_peered":"2024-04-16T15:40:29.919801+0000","last_clean":"2024-04-16T15:40:29.919801+0000","last_became_active":"2024-04-16T15:40:24.536478+0000","last_became_peered":"2024-04-16T15:40:24.536478+0000","last_unstale":"2024-04-16T15:40:29.919801+0000","last_undegraded":"2024-04-16T15:40:29.919801+0000","last_fullsized":"2024-04-16T15:40:29.919801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:26.549813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919917+0000","last_change":"2024-04-16T15:40:23.535794+0000","last_active":"2024-04-16T15:40:29.919917+0000","last_peered":"2024-04-16T15:40:29.919917+0000","last_clean":"2024-04-16T15:40:29.919917+0000","last_became_active":"2024-04-16T15:40:23.535373+0000","last_became_peered":"2024-04-16T15:40:23.535373+0000","last_unstale":"2024-04-16T15:40:29.919917+0000","last_undegraded":"2024-04-16T15:40:29.919917+0000","last_fullsized":"2024-04-16T15:40:29.919917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920113+0000","last_change":"2024-04-16T15:40:28.561348+0000","last_active":"2024-04-16T15:40:29.920113+0000","last_peered":"2024-04-16T15:40:29.920113+0000","last_clean":"2024-04-16T15:40:29.920113+0000","last_became_active":"2024-04-16T15:40:28.560833+0000","last_became_peered":"2024-04-16T15:40:28.560833+0000","last_unstale":"2024-04-16T15:40:29.920113+0000","last_undegraded":"2024-04-16T15:40:29.920113+0000","last_fullsized":"2024-04-16T15:40:29.920113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:22.352368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919131+0000","last_change":"2024-04-16T15:40:24.534724+0000","last_active":"2024-04-16T15:40:29.919131+0000","last_peered":"2024-04-16T15:40:29.919131+0000","last_clean":"2024-04-16T15:40:29.919131+0000","last_became_active":"2024-04-16T15:40:24.534469+0000","last_became_peered":"2024-04-16T15:40:24.534469+0000","last_unstale":"2024-04-16T15:40:29.919131+0000","last_undegraded":"2024-04-16T15:40:29.919131+0000","last_fullsized":"2024-04-16T15:40:29.919131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:50:03.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567368+0000","last_change":"2024-04-16T15:40:23.531841+0000","last_active":"2024-04-16T15:40:29.567368+0000","last_peered":"2024-04-16T15:40:29.567368+0000","last_clean":"2024-04-16T15:40:29.567368+0000","last_became_active":"2024-04-16T15:40:23.531565+0000","last_became_peered":"2024-04-16T15:40:23.531565+0000","last_unstale":"2024-04-16T15:40:29.567368+0000","last_undegraded":"2024-04-16T15:40:29.567368+0000","last_fullsized":"2024-04-16T15:40:29.567368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919600+0000","last_change":"2024-04-16T15:40:28.807348+0000","last_active":"2024-04-16T15:40:29.919600+0000","last_peered":"2024-04-16T15:40:29.919600+0000","last_clean":"2024-04-16T15:40:29.919600+0000","last_became_active":"2024-04-16T15:40:28.806984+0000","last_became_peered":"2024-04-16T15:40:28.806984+0000","last_unstale":"2024-04-16T15:40:29.919600+0000","last_undegraded":"2024-04-16T15:40:29.919600+0000","last_fullsized":"2024-04-16T15:40:29.919600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:30.543430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567817+0000","last_change":"2024-04-16T15:40:24.544401+0000","last_active":"2024-04-16T15:40:29.567817+0000","last_peered":"2024-04-16T15:40:29.567817+0000","last_clean":"2024-04-16T15:40:29.567817+0000","last_became_active":"2024-04-16T15:40:24.544156+0000","last_became_peered":"2024-04-16T15:40:24.544156+0000","last_unstale":"2024-04-16T15:40:29.567817+0000","last_undegraded":"2024-04-16T15:40:29.567817+0000","last_fullsized":"2024-04-16T15:40:29.567817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769950+0000","last_change":"2024-04-16T15:40:23.532974+0000","last_active":"2024-04-16T15:40:29.769950+0000","last_peered":"2024-04-16T15:40:29.769950+0000","last_clean":"2024-04-16T15:40:29.769950+0000","last_became_active":"2024-04-16T15:40:23.532692+0000","last_became_peered":"2024-04-16T15:40:23.532692+0000","last_unstale":"2024-04-16T15:40:29.769950+0000","last_undegraded":"2024-04-16T15:40:29.769950+0000","last_fullsized":"2024-04-16T15:40:29.769950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:19.974713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566451+0000","last_change":"2024-04-16T15:40:28.806764+0000","last_active":"2024-04-16T15:40:29.566451+0000","last_peered":"2024-04-16T15:40:29.566451+0000","last_clean":"2024-04-16T15:40:29.566451+0000","last_became_active":"2024-04-16T15:40:28.806437+0000","last_became_peered":"2024-04-16T15:40:28.806437+0000","last_unstale":"2024-04-16T15:40:29.566451+0000","last_undegraded":"2024-04-16T15:40:29.566451+0000","last_fullsized":"2024-04-16T15:40:29.566451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:52:28.865900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769595+0000","last_change":"2024-04-16T15:40:24.536656+0000","last_active":"2024-04-16T15:40:29.769595+0000","last_peered":"2024-04-16T15:40:29.769595+0000","last_clean":"2024-04-16T15:40:29.769595+0000","last_became_active":"2024-04-16T15:40:24.536405+0000","last_became_peered":"2024-04-16T15:40:24.536405+0000","last_unstale":"2024-04-16T15:40:29.769595+0000","last_undegraded":"2024-04-16T15:40:29.769595+0000","last_fullsized":"2024-04-16T15:40:29.769595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921784+0000","last_change":"2024-04-16T15:40:23.536026+0000","last_active":"2024-04-16T15:40:29.921784+0000","last_peered":"2024-04-16T15:40:29.921784+0000","last_clean":"2024-04-16T15:40:29.921784+0000","last_became_active":"2024-04-16T15:40:23.535220+0000","last_became_peered":"2024-04-16T15:40:23.535220+0000","last_unstale":"2024-04-16T15:40:29.921784+0000","last_undegraded":"2024-04-16T15:40:29.921784+0000","last_fullsized":"2024-04-16T15:40:29.921784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:37.570307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918806+0000","last_change":"2024-04-16T15:40:28.804863+0000","last_active":"2024-04-16T15:40:29.918806+0000","last_peered":"2024-04-16T15:40:29.918806+0000","last_clean":"2024-04-16T15:40:29.918806+0000","last_became_active":"2024-04-16T15:40:28.804502+0000","last_became_peered":"2024-04-16T15:40:28.804502+0000","last_unstale":"2024-04-16T15:40:29.918806+0000","last_undegraded":"2024-04-16T15:40:29.918806+0000","last_fullsized":"2024-04-16T15:40:29.918806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920557+0000","last_change":"2024-04-16T15:40:24.551868+0000","last_active":"2024-04-16T15:40:29.920557+0000","last_peered":"2024-04-16T15:40:29.920557+0000","last_clean":"2024-04-16T15:40:29.920557+0000","last_became_active":"2024-04-16T15:40:24.550950+0000","last_became_peered":"2024-04-16T15:40:24.550950+0000","last_unstale":"2024-04-16T15:40:29.920557+0000","last_undegraded":"2024-04-16T15:40:29.920557+0000","last_fullsized":"2024-04-16T15:40:29.920557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:27.826954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769685+0000","last_change":"2024-04-16T15:40:23.533782+0000","last_active":"2024-04-16T15:40:29.769685+0000","last_peered":"2024-04-16T15:40:29.769685+0000","last_clean":"2024-04-16T15:40:29.769685+0000","last_became_active":"2024-04-16T15:40:23.533547+0000","last_became_peered":"2024-04-16T15:40:23.533547+0000","last_unstale":"2024-04-16T15:40:29.769685+0000","last_undegraded":"2024-04-16T15:40:29.769685+0000","last_fullsized":"2024-04-16T15:40:29.769685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769337+0000","last_change":"2024-04-16T15:40:28.570047+0000","last_active":"2024-04-16T15:40:29.769337+0000","last_peered":"2024-04-16T15:40:29.769337+0000","last_clean":"2024-04-16T15:40:29.769337+0000","last_became_active":"2024-04-16T15:40:28.569688+0000","last_became_peered":"2024-04-16T15:40:28.569688+0000","last_unstale":"2024-04-16T15:40:29.769337+0000","last_undegraded":"2024-04-16T15:40:29.769337+0000","last_fullsized":"2024-04-16T15:40:29.769337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:42.690952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921046+0000","last_change":"2024-04-16T15:40:24.549077+0000","last_active":"2024-04-16T15:40:29.921046+0000","last_peered":"2024-04-16T15:40:29.921046+0000","last_clean":"2024-04-16T15:40:29.921046+0000","last_became_active":"2024-04-16T15:40:24.548802+0000","last_became_peered":"2024-04-16T15:40:24.548802+0000","last_unstale":"2024-04-16T15:40:29.921046+0000","last_undegraded":"2024-04-16T15:40:29.921046+0000","last_fullsized":"2024-04-16T15:40:29.921046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:50.142130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920546+0000","last_change":"2024-04-16T15:40:23.528707+0000","last_active":"2024-04-16T15:40:29.920546+0000","last_peered":"2024-04-16T15:40:29.920546+0000","last_clean":"2024-04-16T15:40:29.920546+0000","last_became_active":"2024-04-16T15:40:23.528426+0000","last_became_peered":"2024-04-16T15:40:23.528426+0000","last_unstale":"2024-04-16T15:40:29.920546+0000","last_undegraded":"2024-04-16T15:40:29.920546+0000","last_fullsized":"2024-04-16T15:40:29.920546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769721+0000","last_change":"2024-04-16T15:40:28.561110+0000","last_active":"2024-04-16T15:40:29.769721+0000","last_peered":"2024-04-16T15:40:29.769721+0000","last_clean":"2024-04-16T15:40:29.769721+0000","last_became_active":"2024-04-16T15:40:28.560719+0000","last_became_peered":"2024-04-16T15:40:28.560719+0000","last_unstale":"2024-04-16T15:40:29.769721+0000","last_undegraded":"2024-04-16T15:40:29.769721+0000","last_fullsized":"2024-04-16T15:40:29.769721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:31.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769261+0000","last_change":"2024-04-16T15:40:24.536757+0000","last_active":"2024-04-16T15:40:29.769261+0000","last_peered":"2024-04-16T15:40:29.769261+0000","last_clean":"2024-04-16T15:40:29.769261+0000","last_became_active":"2024-04-16T15:40:24.536520+0000","last_became_peered":"2024-04-16T15:40:24.536520+0000","last_unstale":"2024-04-16T15:40:29.769261+0000","last_undegraded":"2024-04-16T15:40:29.769261+0000","last_fullsized":"2024-04-16T15:40:29.769261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566917+0000","last_change":"2024-04-16T15:40:23.520551+0000","last_active":"2024-04-16T15:40:29.566917+0000","last_peered":"2024-04-16T15:40:29.566917+0000","last_clean":"2024-04-16T15:40:29.566917+0000","last_became_active":"2024-04-16T15:40:23.520300+0000","last_became_peered":"2024-04-16T15:40:23.520300+0000","last_unstale":"2024-04-16T15:40:29.566917+0000","last_undegraded":"2024-04-16T15:40:29.566917+0000","last_fullsized":"2024-04-16T15:40:29.566917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.770097+0000","last_change":"2024-04-16T15:40:28.570553+0000","last_active":"2024-04-16T15:40:29.770097+0000","last_peered":"2024-04-16T15:40:29.770097+0000","last_clean":"2024-04-16T15:40:29.770097+0000","last_became_active":"2024-04-16T15:40:28.569635+0000","last_became_peered":"2024-04-16T15:40:28.569635+0000","last_unstale":"2024-04-16T15:40:29.770097+0000","last_undegraded":"2024-04-16T15:40:29.770097+0000","last_fullsized":"2024-04-16T15:40:29.770097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:28.407124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918519+0000","last_change":"2024-04-16T15:40:24.537795+0000","last_active":"2024-04-16T15:40:29.918519+0000","last_peered":"2024-04-16T15:40:29.918519+0000","last_clean":"2024-04-16T15:40:29.918519+0000","last_became_active":"2024-04-16T15:40:24.537528+0000","last_became_peered":"2024-04-16T15:40:24.537528+0000","last_unstale":"2024-04-16T15:40:29.918519+0000","last_undegraded":"2024-04-16T15:40:29.918519+0000","last_fullsized":"2024-04-16T15:40:29.918519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:38.574731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920406+0000","last_change":"2024-04-16T15:40:23.533872+0000","last_active":"2024-04-16T15:40:29.920406+0000","last_peered":"2024-04-16T15:40:29.920406+0000","last_clean":"2024-04-16T15:40:29.920406+0000","last_became_active":"2024-04-16T15:40:23.533491+0000","last_became_peered":"2024-04-16T15:40:23.533491+0000","last_unstale":"2024-04-16T15:40:29.920406+0000","last_undegraded":"2024-04-16T15:40:29.920406+0000","last_fullsized":"2024-04-16T15:40:29.920406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:21.943427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919078+0000","last_change":"2024-04-16T15:40:28.807602+0000","last_active":"2024-04-16T15:40:29.919078+0000","last_peered":"2024-04-16T15:40:29.919078+0000","last_clean":"2024-04-16T15:40:29.919078+0000","last_became_active":"2024-04-16T15:40:28.807193+0000","last_became_peered":"2024-04-16T15:40:28.807193+0000","last_unstale":"2024-04-16T15:40:29.919078+0000","last_undegraded":"2024-04-16T15:40:29.919078+0000","last_fullsized":"2024-04-16T15:40:29.919078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:35.057588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567071+0000","last_change":"2024-04-16T15:40:24.546548+0000","last_active":"2024-04-16T15:40:29.567071+0000","last_peered":"2024-04-16T15:40:29.567071+0000","last_clean":"2024-04-16T15:40:29.567071+0000","last_became_active":"2024-04-16T15:40:24.546317+0000","last_became_peered":"2024-04-16T15:40:24.546317+0000","last_unstale":"2024-04-16T15:40:29.567071+0000","last_undegraded":"2024-04-16T15:40:29.567071+0000","last_fullsized":"2024-04-16T15:40:29.567071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:14.076265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921124+0000","last_change":"2024-04-16T15:40:23.529866+0000","last_active":"2024-04-16T15:40:29.921124+0000","last_peered":"2024-04-16T15:40:29.921124+0000","last_clean":"2024-04-16T15:40:29.921124+0000","last_became_active":"2024-04-16T15:40:23.529575+0000","last_became_peered":"2024-04-16T15:40:23.529575+0000","last_unstale":"2024-04-16T15:40:29.921124+0000","last_undegraded":"2024-04-16T15:40:29.921124+0000","last_fullsized":"2024-04-16T15:40:29.921124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920486+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:40:29.920486+0000","last_peered":"2024-04-16T15:40:29.920486+0000","last_clean":"2024-04-16T15:40:29.920486+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:40:29.920486+0000","last_undegraded":"2024-04-16T15:40:29.920486+0000","last_fullsized":"2024-04-16T15:40:29.920486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567180+0000","last_change":"2024-04-16T15:40:28.805311+0000","last_active":"2024-04-16T15:40:29.567180+0000","last_peered":"2024-04-16T15:40:29.567180+0000","last_clean":"2024-04-16T15:40:29.567180+0000","last_became_active":"2024-04-16T15:40:28.804868+0000","last_became_peered":"2024-04-16T15:40:28.804868+0000","last_unstale":"2024-04-16T15:40:29.567180+0000","last_undegraded":"2024-04-16T15:40:29.567180+0000","last_fullsized":"2024-04-16T15:40:29.567180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:32.196316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921580+0000","last_change":"2024-04-16T15:40:24.544114+0000","last_active":"2024-04-16T15:40:29.921580+0000","last_peered":"2024-04-16T15:40:29.921580+0000","last_clean":"2024-04-16T15:40:29.921580+0000","last_became_active":"2024-04-16T15:40:24.543798+0000","last_became_peered":"2024-04-16T15:40:24.543798+0000","last_unstale":"2024-04-16T15:40:29.921580+0000","last_undegraded":"2024-04-16T15:40:29.921580+0000","last_fullsized":"2024-04-16T15:40:29.921580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:38.664081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567401+0000","last_change":"2024-04-16T15:40:23.525608+0000","last_active":"2024-04-16T15:40:29.567401+0000","last_peered":"2024-04-16T15:40:29.567401+0000","last_clean":"2024-04-16T15:40:29.567401+0000","last_became_active":"2024-04-16T15:40:23.525361+0000","last_became_peered":"2024-04-16T15:40:23.525361+0000","last_unstale":"2024-04-16T15:40:29.567401+0000","last_undegraded":"2024-04-16T15:40:29.567401+0000","last_fullsized":"2024-04-16T15:40:29.567401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:17.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566561+0000","last_change":"2024-04-16T15:40:28.806076+0000","last_active":"2024-04-16T15:40:29.566561+0000","last_peered":"2024-04-16T15:40:29.566561+0000","last_clean":"2024-04-16T15:40:29.566561+0000","last_became_active":"2024-04-16T15:40:28.805387+0000","last_became_peered":"2024-04-16T15:40:28.805387+0000","last_unstale":"2024-04-16T15:40:29.566561+0000","last_undegraded":"2024-04-16T15:40:29.566561+0000","last_fullsized":"2024-04-16T15:40:29.566561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:55.765984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920030+0000","last_change":"2024-04-16T15:40:24.546430+0000","last_active":"2024-04-16T15:40:29.920030+0000","last_peered":"2024-04-16T15:40:29.920030+0000","last_clean":"2024-04-16T15:40:29.920030+0000","last_became_active":"2024-04-16T15:40:24.546177+0000","last_became_peered":"2024-04-16T15:40:24.546177+0000","last_unstale":"2024-04-16T15:40:29.920030+0000","last_undegraded":"2024-04-16T15:40:29.920030+0000","last_fullsized":"2024-04-16T15:40:29.920030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:00.952582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769778+0000","last_change":"2024-04-16T15:40:23.537830+0000","last_active":"2024-04-16T15:40:29.769778+0000","last_peered":"2024-04-16T15:40:29.769778+0000","last_clean":"2024-04-16T15:40:29.769778+0000","last_became_active":"2024-04-16T15:40:23.537574+0000","last_became_peered":"2024-04-16T15:40:23.537574+0000","last_unstale":"2024-04-16T15:40:29.769778+0000","last_undegraded":"2024-04-16T15:40:29.769778+0000","last_fullsized":"2024-04-16T15:40:29.769778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918912+0000","last_change":"2024-04-16T15:40:28.571485+0000","last_active":"2024-04-16T15:40:29.918912+0000","last_peered":"2024-04-16T15:40:29.918912+0000","last_clean":"2024-04-16T15:40:29.918912+0000","last_became_active":"2024-04-16T15:40:28.570667+0000","last_became_peered":"2024-04-16T15:40:28.570667+0000","last_unstale":"2024-04-16T15:40:29.918912+0000","last_undegraded":"2024-04-16T15:40:29.918912+0000","last_fullsized":"2024-04-16T15:40:29.918912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:04.979453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920895+0000","last_change":"2024-04-16T15:40:24.548267+0000","last_active":"2024-04-16T15:40:29.920895+0000","last_peered":"2024-04-16T15:40:29.920895+0000","last_clean":"2024-04-16T15:40:29.920895+0000","last_became_active":"2024-04-16T15:40:24.547973+0000","last_became_peered":"2024-04-16T15:40:24.547973+0000","last_unstale":"2024-04-16T15:40:29.920895+0000","last_undegraded":"2024-04-16T15:40:29.920895+0000","last_fullsized":"2024-04-16T15:40:29.920895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:50.186857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566718+0000","last_change":"2024-04-16T15:40:23.523969+0000","last_active":"2024-04-16T15:40:29.566718+0000","last_peered":"2024-04-16T15:40:29.566718+0000","last_clean":"2024-04-16T15:40:29.566718+0000","last_became_active":"2024-04-16T15:40:23.523676+0000","last_became_peered":"2024-04-16T15:40:23.523676+0000","last_unstale":"2024-04-16T15:40:29.566718+0000","last_undegraded":"2024-04-16T15:40:29.566718+0000","last_fullsized":"2024-04-16T15:40:29.566718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919411+0000","last_change":"2024-04-16T15:40:28.807519+0000","last_active":"2024-04-16T15:40:29.919411+0000","last_peered":"2024-04-16T15:40:29.919411+0000","last_clean":"2024-04-16T15:40:29.919411+0000","last_became_active":"2024-04-16T15:40:28.807122+0000","last_became_peered":"2024-04-16T15:40:28.807122+0000","last_unstale":"2024-04-16T15:40:29.919411+0000","last_undegraded":"2024-04-16T15:40:29.919411+0000","last_fullsized":"2024-04-16T15:40:29.919411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:52:28.865900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920489+0000","last_change":"2024-04-16T15:40:24.545147+0000","last_active":"2024-04-16T15:40:29.920489+0000","last_peered":"2024-04-16T15:40:29.920489+0000","last_clean":"2024-04-16T15:40:29.920489+0000","last_became_active":"2024-04-16T15:40:24.544857+0000","last_became_peered":"2024-04-16T15:40:24.544857+0000","last_unstale":"2024-04-16T15:40:29.920489+0000","last_undegraded":"2024-04-16T15:40:29.920489+0000","last_fullsized":"2024-04-16T15:40:29.920489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:31.030716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920619+0000","last_change":"2024-04-16T15:40:23.536928+0000","last_active":"2024-04-16T15:40:29.920619+0000","last_peered":"2024-04-16T15:40:29.920619+0000","last_clean":"2024-04-16T15:40:29.920619+0000","last_became_active":"2024-04-16T15:40:23.535290+0000","last_became_peered":"2024-04-16T15:40:23.535290+0000","last_unstale":"2024-04-16T15:40:29.920619+0000","last_undegraded":"2024-04-16T15:40:29.920619+0000","last_fullsized":"2024-04-16T15:40:29.920619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:26.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919286+0000","last_change":"2024-04-16T15:40:28.561919+0000","last_active":"2024-04-16T15:40:29.919286+0000","last_peered":"2024-04-16T15:40:29.919286+0000","last_clean":"2024-04-16T15:40:29.919286+0000","last_became_active":"2024-04-16T15:40:28.561491+0000","last_became_peered":"2024-04-16T15:40:28.561491+0000","last_unstale":"2024-04-16T15:40:29.919286+0000","last_undegraded":"2024-04-16T15:40:29.919286+0000","last_fullsized":"2024-04-16T15:40:29.919286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:19.925886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919818+0000","last_change":"2024-04-16T15:40:24.544255+0000","last_active":"2024-04-16T15:40:29.919818+0000","last_peered":"2024-04-16T15:40:29.919818+0000","last_clean":"2024-04-16T15:40:29.919818+0000","last_became_active":"2024-04-16T15:40:24.543995+0000","last_became_peered":"2024-04-16T15:40:24.543995+0000","last_unstale":"2024-04-16T15:40:29.919818+0000","last_undegraded":"2024-04-16T15:40:29.919818+0000","last_fullsized":"2024-04-16T15:40:29.919818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:28.169445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566390+0000","last_change":"2024-04-16T15:40:23.530042+0000","last_active":"2024-04-16T15:40:29.566390+0000","last_peered":"2024-04-16T15:40:29.566390+0000","last_clean":"2024-04-16T15:40:29.566390+0000","last_became_active":"2024-04-16T15:40:23.528762+0000","last_became_peered":"2024-04-16T15:40:23.528762+0000","last_unstale":"2024-04-16T15:40:29.566390+0000","last_undegraded":"2024-04-16T15:40:29.566390+0000","last_fullsized":"2024-04-16T15:40:29.566390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919066+0000","last_change":"2024-04-16T15:40:28.568830+0000","last_active":"2024-04-16T15:40:29.919066+0000","last_peered":"2024-04-16T15:40:29.919066+0000","last_clean":"2024-04-16T15:40:29.919066+0000","last_became_active":"2024-04-16T15:40:28.568395+0000","last_became_peered":"2024-04-16T15:40:28.568395+0000","last_unstale":"2024-04-16T15:40:29.919066+0000","last_undegraded":"2024-04-16T15:40:29.919066+0000","last_fullsized":"2024-04-16T15:40:29.919066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:42.690952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566645+0000","last_change":"2024-04-16T15:40:24.543975+0000","last_active":"2024-04-16T15:40:29.566645+0000","last_peered":"2024-04-16T15:40:29.566645+0000","last_clean":"2024-04-16T15:40:29.566645+0000","last_became_active":"2024-04-16T15:40:24.543720+0000","last_became_peered":"2024-04-16T15:40:24.543720+0000","last_unstale":"2024-04-16T15:40:29.566645+0000","last_undegraded":"2024-04-16T15:40:29.566645+0000","last_fullsized":"2024-04-16T15:40:29.566645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566771+0000","last_change":"2024-04-16T15:40:23.536758+0000","last_active":"2024-04-16T15:40:29.566771+0000","last_peered":"2024-04-16T15:40:29.566771+0000","last_clean":"2024-04-16T15:40:29.566771+0000","last_became_active":"2024-04-16T15:40:23.536479+0000","last_became_peered":"2024-04-16T15:40:23.536479+0000","last_unstale":"2024-04-16T15:40:29.566771+0000","last_undegraded":"2024-04-16T15:40:29.566771+0000","last_fullsized":"2024-04-16T15:40:29.566771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567798+0000","last_change":"2024-04-16T15:40:28.807067+0000","last_active":"2024-04-16T15:40:29.567798+0000","last_peered":"2024-04-16T15:40:29.567798+0000","last_clean":"2024-04-16T15:40:29.567798+0000","last_became_active":"2024-04-16T15:40:28.805886+0000","last_became_peered":"2024-04-16T15:40:28.805886+0000","last_unstale":"2024-04-16T15:40:29.567798+0000","last_undegraded":"2024-04-16T15:40:29.567798+0000","last_fullsized":"2024-04-16T15:40:29.567798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:52.503196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920077+0000","last_change":"2024-04-16T15:40:24.546841+0000","last_active":"2024-04-16T15:40:29.920077+0000","last_peered":"2024-04-16T15:40:29.920077+0000","last_clean":"2024-04-16T15:40:29.920077+0000","last_became_active":"2024-04-16T15:40:24.546578+0000","last_became_peered":"2024-04-16T15:40:24.546578+0000","last_unstale":"2024-04-16T15:40:29.920077+0000","last_undegraded":"2024-04-16T15:40:29.920077+0000","last_fullsized":"2024-04-16T15:40:29.920077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:00.952582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920432+0000","last_change":"2024-04-16T15:40:23.527740+0000","last_active":"2024-04-16T15:40:29.920432+0000","last_peered":"2024-04-16T15:40:29.920432+0000","last_clean":"2024-04-16T15:40:29.920432+0000","last_became_active":"2024-04-16T15:40:23.527465+0000","last_became_peered":"2024-04-16T15:40:23.527465+0000","last_unstale":"2024-04-16T15:40:29.920432+0000","last_undegraded":"2024-04-16T15:40:29.920432+0000","last_fullsized":"2024-04-16T15:40:29.920432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567222+0000","last_change":"2024-04-16T15:40:28.804586+0000","last_active":"2024-04-16T15:40:29.567222+0000","last_peered":"2024-04-16T15:40:29.567222+0000","last_clean":"2024-04-16T15:40:29.567222+0000","last_became_active":"2024-04-16T15:40:28.804188+0000","last_became_peered":"2024-04-16T15:40:28.804188+0000","last_unstale":"2024-04-16T15:40:29.567222+0000","last_undegraded":"2024-04-16T15:40:29.567222+0000","last_fullsized":"2024-04-16T15:40:29.567222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:54:44.467991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918678+0000","last_change":"2024-04-16T15:40:24.548554+0000","last_active":"2024-04-16T15:40:29.918678+0000","last_peered":"2024-04-16T15:40:29.918678+0000","last_clean":"2024-04-16T15:40:29.918678+0000","last_became_active":"2024-04-16T15:40:24.548292+0000","last_became_peered":"2024-04-16T15:40:24.548292+0000","last_unstale":"2024-04-16T15:40:29.918678+0000","last_undegraded":"2024-04-16T15:40:29.918678+0000","last_fullsized":"2024-04-16T15:40:29.918678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918961+0000","last_change":"2024-04-16T15:40:23.537872+0000","last_active":"2024-04-16T15:40:29.918961+0000","last_peered":"2024-04-16T15:40:29.918961+0000","last_clean":"2024-04-16T15:40:29.918961+0000","last_became_active":"2024-04-16T15:40:23.537586+0000","last_became_peered":"2024-04-16T15:40:23.537586+0000","last_unstale":"2024-04-16T15:40:29.918961+0000","last_undegraded":"2024-04-16T15:40:29.918961+0000","last_fullsized":"2024-04-16T15:40:29.918961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:35.227712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920389+0000","last_change":"2024-04-16T15:40:28.805253+0000","last_active":"2024-04-16T15:40:29.920389+0000","last_peered":"2024-04-16T15:40:29.920389+0000","last_clean":"2024-04-16T15:40:29.920389+0000","last_became_active":"2024-04-16T15:40:28.804927+0000","last_became_peered":"2024-04-16T15:40:28.804927+0000","last_unstale":"2024-04-16T15:40:29.920389+0000","last_undegraded":"2024-04-16T15:40:29.920389+0000","last_fullsized":"2024-04-16T15:40:29.920389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:55.036666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921526+0000","last_change":"2024-04-16T15:40:24.549880+0000","last_active":"2024-04-16T15:40:29.921526+0000","last_peered":"2024-04-16T15:40:29.921526+0000","last_clean":"2024-04-16T15:40:29.921526+0000","last_became_active":"2024-04-16T15:40:24.549176+0000","last_became_peered":"2024-04-16T15:40:24.549176+0000","last_unstale":"2024-04-16T15:40:29.921526+0000","last_undegraded":"2024-04-16T15:40:29.921526+0000","last_fullsized":"2024-04-16T15:40:29.921526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:32.578262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:34.090660+0000","last_change":"2024-04-16T15:40:23.528022+0000","last_active":"2024-04-16T15:40:34.090660+0000","last_peered":"2024-04-16T15:40:34.090660+0000","last_clean":"2024-04-16T15:40:34.090660+0000","last_became_active":"2024-04-16T15:40:23.527723+0000","last_became_peered":"2024-04-16T15:40:23.527723+0000","last_unstale":"2024-04-16T15:40:34.090660+0000","last_undegraded":"2024-04-16T15:40:34.090660+0000","last_fullsized":"2024-04-16T15:40:34.090660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769345+0000","last_change":"2024-04-16T15:40:28.570589+0000","last_active":"2024-04-16T15:40:29.769345+0000","last_peered":"2024-04-16T15:40:29.769345+0000","last_clean":"2024-04-16T15:40:29.769345+0000","last_became_active":"2024-04-16T15:40:28.570223+0000","last_became_peered":"2024-04-16T15:40:28.570223+0000","last_unstale":"2024-04-16T15:40:29.769345+0000","last_undegraded":"2024-04-16T15:40:29.769345+0000","last_fullsized":"2024-04-16T15:40:29.769345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:35.057588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920264+0000","last_change":"2024-04-16T15:40:24.548751+0000","last_active":"2024-04-16T15:40:29.920264+0000","last_peered":"2024-04-16T15:40:29.920264+0000","last_clean":"2024-04-16T15:40:29.920264+0000","last_became_active":"2024-04-16T15:40:24.548459+0000","last_became_peered":"2024-04-16T15:40:24.548459+0000","last_unstale":"2024-04-16T15:40:29.920264+0000","last_undegraded":"2024-04-16T15:40:29.920264+0000","last_fullsized":"2024-04-16T15:40:29.920264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:22.947852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769687+0000","last_change":"2024-04-16T15:40:23.531085+0000","last_active":"2024-04-16T15:40:29.769687+0000","last_peered":"2024-04-16T15:40:29.769687+0000","last_clean":"2024-04-16T15:40:29.769687+0000","last_became_active":"2024-04-16T15:40:23.530814+0000","last_became_peered":"2024-04-16T15:40:23.530814+0000","last_unstale":"2024-04-16T15:40:29.769687+0000","last_undegraded":"2024-04-16T15:40:29.769687+0000","last_fullsized":"2024-04-16T15:40:29.769687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920980+0000","last_change":"2024-04-16T15:40:28.570865+0000","last_active":"2024-04-16T15:40:29.920980+0000","last_peered":"2024-04-16T15:40:29.920980+0000","last_clean":"2024-04-16T15:40:29.920980+0000","last_became_active":"2024-04-16T15:40:28.570475+0000","last_became_peered":"2024-04-16T15:40:28.570475+0000","last_unstale":"2024-04-16T15:40:29.920980+0000","last_undegraded":"2024-04-16T15:40:29.920980+0000","last_fullsized":"2024-04-16T15:40:29.920980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:32.196316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919181+0000","last_change":"2024-04-16T15:40:24.543283+0000","last_active":"2024-04-16T15:40:29.919181+0000","last_peered":"2024-04-16T15:40:29.919181+0000","last_clean":"2024-04-16T15:40:29.919181+0000","last_became_active":"2024-04-16T15:40:24.542708+0000","last_became_peered":"2024-04-16T15:40:24.542708+0000","last_unstale":"2024-04-16T15:40:29.919181+0000","last_undegraded":"2024-04-16T15:40:29.919181+0000","last_fullsized":"2024-04-16T15:40:29.919181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920195+0000","last_change":"2024-04-16T15:40:23.530402+0000","last_active":"2024-04-16T15:40:29.920195+0000","last_peered":"2024-04-16T15:40:29.920195+0000","last_clean":"2024-04-16T15:40:29.920195+0000","last_became_active":"2024-04-16T15:40:23.530122+0000","last_became_peered":"2024-04-16T15:40:23.530122+0000","last_unstale":"2024-04-16T15:40:29.920195+0000","last_undegraded":"2024-04-16T15:40:29.920195+0000","last_fullsized":"2024-04-16T15:40:29.920195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566298+0000","last_change":"2024-04-16T15:40:28.563586+0000","last_active":"2024-04-16T15:40:29.566298+0000","last_peered":"2024-04-16T15:40:29.566298+0000","last_clean":"2024-04-16T15:40:29.566298+0000","last_became_active":"2024-04-16T15:40:28.563187+0000","last_became_peered":"2024-04-16T15:40:28.563187+0000","last_unstale":"2024-04-16T15:40:29.566298+0000","last_undegraded":"2024-04-16T15:40:29.566298+0000","last_fullsized":"2024-04-16T15:40:29.566298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:28.407124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919923+0000","last_change":"2024-04-16T15:40:24.550624+0000","last_active":"2024-04-16T15:40:29.919923+0000","last_peered":"2024-04-16T15:40:29.919923+0000","last_clean":"2024-04-16T15:40:29.919923+0000","last_became_active":"2024-04-16T15:40:24.550410+0000","last_became_peered":"2024-04-16T15:40:24.550410+0000","last_unstale":"2024-04-16T15:40:29.919923+0000","last_undegraded":"2024-04-16T15:40:29.919923+0000","last_fullsized":"2024-04-16T15:40:29.919923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:32.578262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567285+0000","last_change":"2024-04-16T15:40:23.531043+0000","last_active":"2024-04-16T15:40:29.567285+0000","last_peered":"2024-04-16T15:40:29.567285+0000","last_clean":"2024-04-16T15:40:29.567285+0000","last_became_active":"2024-04-16T15:40:23.530764+0000","last_became_peered":"2024-04-16T15:40:23.530764+0000","last_unstale":"2024-04-16T15:40:29.567285+0000","last_undegraded":"2024-04-16T15:40:29.567285+0000","last_fullsized":"2024-04-16T15:40:29.567285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566507+0000","last_change":"2024-04-16T15:40:28.805696+0000","last_active":"2024-04-16T15:40:29.566507+0000","last_peered":"2024-04-16T15:40:29.566507+0000","last_clean":"2024-04-16T15:40:29.566507+0000","last_became_active":"2024-04-16T15:40:28.805319+0000","last_became_peered":"2024-04-16T15:40:28.805319+0000","last_unstale":"2024-04-16T15:40:29.566507+0000","last_undegraded":"2024-04-16T15:40:29.566507+0000","last_fullsized":"2024-04-16T15:40:29.566507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:55.036666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918914+0000","last_change":"2024-04-16T15:40:28.804646+0000","last_active":"2024-04-16T15:40:29.918914+0000","last_peered":"2024-04-16T15:40:29.918914+0000","last_clean":"2024-04-16T15:40:29.918914+0000","last_became_active":"2024-04-16T15:40:28.804239+0000","last_became_peered":"2024-04-16T15:40:28.804239+0000","last_unstale":"2024-04-16T15:40:29.918914+0000","last_undegraded":"2024-04-16T15:40:29.918914+0000","last_fullsized":"2024-04-16T15:40:29.918914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:36.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769103+0000","last_change":"2024-04-16T15:40:24.533204+0000","last_active":"2024-04-16T15:40:29.769103+0000","last_peered":"2024-04-16T15:40:29.769103+0000","last_clean":"2024-04-16T15:40:29.769103+0000","last_became_active":"2024-04-16T15:40:24.532947+0000","last_became_peered":"2024-04-16T15:40:24.532947+0000","last_unstale":"2024-04-16T15:40:29.769103+0000","last_undegraded":"2024-04-16T15:40:29.769103+0000","last_fullsized":"2024-04-16T15:40:29.769103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919749+0000","last_change":"2024-04-16T15:40:23.537693+0000","last_active":"2024-04-16T15:40:29.919749+0000","last_peered":"2024-04-16T15:40:29.919749+0000","last_clean":"2024-04-16T15:40:29.919749+0000","last_became_active":"2024-04-16T15:40:23.536396+0000","last_became_peered":"2024-04-16T15:40:23.536396+0000","last_unstale":"2024-04-16T15:40:29.919749+0000","last_undegraded":"2024-04-16T15:40:29.919749+0000","last_fullsized":"2024-04-16T15:40:29.919749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567324+0000","last_change":"2024-04-16T15:40:28.567889+0000","last_active":"2024-04-16T15:40:29.567324+0000","last_peered":"2024-04-16T15:40:29.567324+0000","last_clean":"2024-04-16T15:40:29.567324+0000","last_became_active":"2024-04-16T15:40:28.567519+0000","last_became_peered":"2024-04-16T15:40:28.567519+0000","last_unstale":"2024-04-16T15:40:29.567324+0000","last_undegraded":"2024-04-16T15:40:29.567324+0000","last_fullsized":"2024-04-16T15:40:29.567324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566216+0000","last_change":"2024-04-16T15:40:24.550785+0000","last_active":"2024-04-16T15:40:29.566216+0000","last_peered":"2024-04-16T15:40:29.566216+0000","last_clean":"2024-04-16T15:40:29.566216+0000","last_became_active":"2024-04-16T15:40:24.550492+0000","last_became_peered":"2024-04-16T15:40:24.550492+0000","last_unstale":"2024-04-16T15:40:29.566216+0000","last_undegraded":"2024-04-16T15:40:29.566216+0000","last_fullsized":"2024-04-16T15:40:29.566216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567393+0000","last_change":"2024-04-16T15:40:23.527649+0000","last_active":"2024-04-16T15:40:29.567393+0000","last_peered":"2024-04-16T15:40:29.567393+0000","last_clean":"2024-04-16T15:40:29.567393+0000","last_became_active":"2024-04-16T15:40:23.526805+0000","last_became_peered":"2024-04-16T15:40:23.526805+0000","last_unstale":"2024-04-16T15:40:29.567393+0000","last_undegraded":"2024-04-16T15:40:29.567393+0000","last_fullsized":"2024-04-16T15:40:29.567393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:17.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566914+0000","last_change":"2024-04-16T15:40:23.536732+0000","last_active":"2024-04-16T15:40:29.566914+0000","last_peered":"2024-04-16T15:40:29.566914+0000","last_clean":"2024-04-16T15:40:29.566914+0000","last_became_active":"2024-04-16T15:40:23.536431+0000","last_became_peered":"2024-04-16T15:40:23.536431+0000","last_unstale":"2024-04-16T15:40:29.566914+0000","last_undegraded":"2024-04-16T15:40:29.566914+0000","last_fullsized":"2024-04-16T15:40:29.566914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918955+0000","last_change":"2024-04-16T15:40:24.538554+0000","last_active":"2024-04-16T15:40:29.918955+0000","last_peered":"2024-04-16T15:40:29.918955+0000","last_clean":"2024-04-16T15:40:29.918955+0000","last_became_active":"2024-04-16T15:40:24.538271+0000","last_became_peered":"2024-04-16T15:40:24.538271+0000","last_unstale":"2024-04-16T15:40:29.918955+0000","last_undegraded":"2024-04-16T15:40:29.918955+0000","last_fullsized":"2024-04-16T15:40:29.918955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567207+0000","last_change":"2024-04-16T15:40:28.566595+0000","last_active":"2024-04-16T15:40:29.567207+0000","last_peered":"2024-04-16T15:40:29.567207+0000","last_clean":"2024-04-16T15:40:29.567207+0000","last_became_active":"2024-04-16T15:40:28.566177+0000","last_became_peered":"2024-04-16T15:40:28.566177+0000","last_unstale":"2024-04-16T15:40:29.567207+0000","last_undegraded":"2024-04-16T15:40:29.567207+0000","last_fullsized":"2024-04-16T15:40:29.567207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:34.092734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567347+0000","last_change":"2024-04-16T15:40:23.528855+0000","last_active":"2024-04-16T15:40:29.567347+0000","last_peered":"2024-04-16T15:40:29.567347+0000","last_clean":"2024-04-16T15:40:29.567347+0000","last_became_active":"2024-04-16T15:40:23.528616+0000","last_became_peered":"2024-04-16T15:40:23.528616+0000","last_unstale":"2024-04-16T15:40:29.567347+0000","last_undegraded":"2024-04-16T15:40:29.567347+0000","last_fullsized":"2024-04-16T15:40:29.567347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919604+0000","last_change":"2024-04-16T15:40:24.535332+0000","last_active":"2024-04-16T15:40:29.919604+0000","last_peered":"2024-04-16T15:40:29.919604+0000","last_clean":"2024-04-16T15:40:29.919604+0000","last_became_active":"2024-04-16T15:40:24.535031+0000","last_became_peered":"2024-04-16T15:40:24.535031+0000","last_unstale":"2024-04-16T15:40:29.919604+0000","last_undegraded":"2024-04-16T15:40:29.919604+0000","last_fullsized":"2024-04-16T15:40:29.919604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:45.872510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567748+0000","last_change":"2024-04-16T15:40:28.564309+0000","last_active":"2024-04-16T15:40:29.567748+0000","last_peered":"2024-04-16T15:40:29.567748+0000","last_clean":"2024-04-16T15:40:29.567748+0000","last_became_active":"2024-04-16T15:40:28.563916+0000","last_became_peered":"2024-04-16T15:40:28.563916+0000","last_unstale":"2024-04-16T15:40:29.567748+0000","last_undegraded":"2024-04-16T15:40:29.567748+0000","last_fullsized":"2024-04-16T15:40:29.567748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:02:28.301124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566423+0000","last_change":"2024-04-16T15:40:24.543242+0000","last_active":"2024-04-16T15:40:29.566423+0000","last_peered":"2024-04-16T15:40:29.566423+0000","last_clean":"2024-04-16T15:40:29.566423+0000","last_became_active":"2024-04-16T15:40:24.542947+0000","last_became_peered":"2024-04-16T15:40:24.542947+0000","last_unstale":"2024-04-16T15:40:29.566423+0000","last_undegraded":"2024-04-16T15:40:29.566423+0000","last_fullsized":"2024-04-16T15:40:29.566423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566822+0000","last_change":"2024-04-16T15:40:24.535196+0000","last_active":"2024-04-16T15:40:29.566822+0000","last_peered":"2024-04-16T15:40:29.566822+0000","last_clean":"2024-04-16T15:40:29.566822+0000","last_became_active":"2024-04-16T15:40:24.534885+0000","last_became_peered":"2024-04-16T15:40:24.534885+0000","last_unstale":"2024-04-16T15:40:29.566822+0000","last_undegraded":"2024-04-16T15:40:29.566822+0000","last_fullsized":"2024-04-16T15:40:29.566822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:20.979138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920587+0000","last_change":"2024-04-16T15:40:24.548666+0000","last_active":"2024-04-16T15:40:29.920587+0000","last_peered":"2024-04-16T15:40:29.920587+0000","last_clean":"2024-04-16T15:40:29.920587+0000","last_became_active":"2024-04-16T15:40:24.548388+0000","last_became_peered":"2024-04-16T15:40:24.548388+0000","last_unstale":"2024-04-16T15:40:29.920587+0000","last_undegraded":"2024-04-16T15:40:29.920587+0000","last_fullsized":"2024-04-16T15:40:29.920587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:28.169445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918853+0000","last_change":"2024-04-16T15:40:24.542842+0000","last_active":"2024-04-16T15:40:29.918853+0000","last_peered":"2024-04-16T15:40:29.918853+0000","last_clean":"2024-04-16T15:40:29.918853+0000","last_became_active":"2024-04-16T15:40:24.542514+0000","last_became_peered":"2024-04-16T15:40:24.542514+0000","last_unstale":"2024-04-16T15:40:29.918853+0000","last_undegraded":"2024-04-16T15:40:29.918853+0000","last_fullsized":"2024-04-16T15:40:29.918853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567390+0000","last_change":"2024-04-16T15:40:24.543863+0000","last_active":"2024-04-16T15:40:29.567390+0000","last_peered":"2024-04-16T15:40:29.567390+0000","last_clean":"2024-04-16T15:40:29.567390+0000","last_became_active":"2024-04-16T15:40:24.543611+0000","last_became_peered":"2024-04-16T15:40:24.543611+0000","last_unstale":"2024-04-16T15:40:29.567390+0000","last_undegraded":"2024-04-16T15:40:29.567390+0000","last_fullsized":"2024-04-16T15:40:29.567390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769571+0000","last_change":"2024-04-16T15:40:24.533049+0000","last_active":"2024-04-16T15:40:29.769571+0000","last_peered":"2024-04-16T15:40:29.769571+0000","last_clean":"2024-04-16T15:40:29.769571+0000","last_became_active":"2024-04-16T15:40:24.532782+0000","last_became_peered":"2024-04-16T15:40:24.532782+0000","last_unstale":"2024-04-16T15:40:29.769571+0000","last_undegraded":"2024-04-16T15:40:29.769571+0000","last_fullsized":"2024-04-16T15:40:29.769571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567235+0000","last_change":"2024-04-16T15:40:24.546110+0000","last_active":"2024-04-16T15:40:29.567235+0000","last_peered":"2024-04-16T15:40:29.567235+0000","last_clean":"2024-04-16T15:40:29.567235+0000","last_became_active":"2024-04-16T15:40:24.545660+0000","last_became_peered":"2024-04-16T15:40:24.545660+0000","last_unstale":"2024-04-16T15:40:29.567235+0000","last_undegraded":"2024-04-16T15:40:29.567235+0000","last_fullsized":"2024-04-16T15:40:29.567235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567701+0000","last_change":"2024-04-16T15:40:24.543839+0000","last_active":"2024-04-16T15:40:29.567701+0000","last_peered":"2024-04-16T15:40:29.567701+0000","last_clean":"2024-04-16T15:40:29.567701+0000","last_became_active":"2024-04-16T15:40:24.543560+0000","last_became_peered":"2024-04-16T15:40:24.543560+0000","last_unstale":"2024-04-16T15:40:29.567701+0000","last_undegraded":"2024-04-16T15:40:29.567701+0000","last_fullsized":"2024-04-16T15:40:29.567701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919650+0000","last_change":"2024-04-16T15:40:24.533578+0000","last_active":"2024-04-16T15:40:29.919650+0000","last_peered":"2024-04-16T15:40:29.919650+0000","last_clean":"2024-04-16T15:40:29.919650+0000","last_became_active":"2024-04-16T15:40:24.533335+0000","last_became_peered":"2024-04-16T15:40:24.533335+0000","last_unstale":"2024-04-16T15:40:29.919650+0000","last_undegraded":"2024-04-16T15:40:29.919650+0000","last_fullsized":"2024-04-16T15:40:29.919650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567192+0000","last_change":"2024-04-16T15:40:24.545340+0000","last_active":"2024-04-16T15:40:29.567192+0000","last_peered":"2024-04-16T15:40:29.567192+0000","last_clean":"2024-04-16T15:40:29.567192+0000","last_became_active":"2024-04-16T15:40:24.545096+0000","last_became_peered":"2024-04-16T15:40:24.545096+0000","last_unstale":"2024-04-16T15:40:29.567192+0000","last_undegraded":"2024-04-16T15:40:29.567192+0000","last_fullsized":"2024-04-16T15:40:29.567192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920042+0000","last_change":"2024-04-16T15:40:24.551716+0000","last_active":"2024-04-16T15:40:29.920042+0000","last_peered":"2024-04-16T15:40:29.920042+0000","last_clean":"2024-04-16T15:40:29.920042+0000","last_became_active":"2024-04-16T15:40:24.551360+0000","last_became_peered":"2024-04-16T15:40:24.551360+0000","last_unstale":"2024-04-16T15:40:29.920042+0000","last_undegraded":"2024-04-16T15:40:29.920042+0000","last_fullsized":"2024-04-16T15:40:29.920042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:14.076265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918705+0000","last_change":"2024-04-16T15:40:24.549554+0000","last_active":"2024-04-16T15:40:29.918705+0000","last_peered":"2024-04-16T15:40:29.918705+0000","last_clean":"2024-04-16T15:40:29.918705+0000","last_became_active":"2024-04-16T15:40:24.548848+0000","last_became_peered":"2024-04-16T15:40:24.548848+0000","last_unstale":"2024-04-16T15:40:29.918705+0000","last_undegraded":"2024-04-16T15:40:29.918705+0000","last_fullsized":"2024-04-16T15:40:29.918705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567025+0000","last_change":"2024-04-16T15:40:24.539612+0000","last_active":"2024-04-16T15:40:29.567025+0000","last_peered":"2024-04-16T15:40:29.567025+0000","last_clean":"2024-04-16T15:40:29.567025+0000","last_became_active":"2024-04-16T15:40:24.536911+0000","last_became_peered":"2024-04-16T15:40:24.536911+0000","last_unstale":"2024-04-16T15:40:29.567025+0000","last_undegraded":"2024-04-16T15:40:29.567025+0000","last_fullsized":"2024-04-16T15:40:29.567025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921716+0000","last_change":"2024-04-16T15:40:24.550620+0000","last_active":"2024-04-16T15:40:29.921716+0000","last_peered":"2024-04-16T15:40:29.921716+0000","last_clean":"2024-04-16T15:40:29.921716+0000","last_became_active":"2024-04-16T15:40:24.549587+0000","last_became_peered":"2024-04-16T15:40:24.549587+0000","last_unstale":"2024-04-16T15:40:29.921716+0000","last_undegraded":"2024-04-16T15:40:29.921716+0000","last_fullsized":"2024-04-16T15:40:29.921716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:27.826954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566610+0000","last_change":"2024-04-16T15:40:24.539758+0000","last_active":"2024-04-16T15:40:29.566610+0000","last_peered":"2024-04-16T15:40:29.566610+0000","last_clean":"2024-04-16T15:40:29.566610+0000","last_became_active":"2024-04-16T15:40:24.539525+0000","last_became_peered":"2024-04-16T15:40:24.539525+0000","last_unstale":"2024-04-16T15:40:29.566610+0000","last_undegraded":"2024-04-16T15:40:29.566610+0000","last_fullsized":"2024-04-16T15:40:29.566610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:44.286135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921033+0000","last_change":"2024-04-16T15:40:24.548867+0000","last_active":"2024-04-16T15:40:29.921033+0000","last_peered":"2024-04-16T15:40:29.921033+0000","last_clean":"2024-04-16T15:40:29.921033+0000","last_became_active":"2024-04-16T15:40:24.548585+0000","last_became_peered":"2024-04-16T15:40:24.548585+0000","last_unstale":"2024-04-16T15:40:29.921033+0000","last_undegraded":"2024-04-16T15:40:29.921033+0000","last_fullsized":"2024-04-16T15:40:29.921033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:31.030716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919546+0000","last_change":"2024-04-16T15:40:24.545697+0000","last_active":"2024-04-16T15:40:29.919546+0000","last_peered":"2024-04-16T15:40:29.919546+0000","last_clean":"2024-04-16T15:40:29.919546+0000","last_became_active":"2024-04-16T15:40:24.545409+0000","last_became_peered":"2024-04-16T15:40:24.545409+0000","last_unstale":"2024-04-16T15:40:29.919546+0000","last_undegraded":"2024-04-16T15:40:29.919546+0000","last_fullsized":"2024-04-16T15:40:29.919546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769173+0000","last_change":"2024-04-16T15:40:24.544806+0000","last_active":"2024-04-16T15:40:29.769173+0000","last_peered":"2024-04-16T15:40:29.769173+0000","last_clean":"2024-04-16T15:40:29.769173+0000","last_became_active":"2024-04-16T15:40:24.544550+0000","last_became_peered":"2024-04-16T15:40:24.544550+0000","last_unstale":"2024-04-16T15:40:29.769173+0000","last_undegraded":"2024-04-16T15:40:29.769173+0000","last_fullsized":"2024-04-16T15:40:29.769173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918672+0000","last_change":"2024-04-16T15:40:24.542813+0000","last_active":"2024-04-16T15:40:29.918672+0000","last_peered":"2024-04-16T15:40:29.918672+0000","last_clean":"2024-04-16T15:40:29.918672+0000","last_became_active":"2024-04-16T15:40:24.534241+0000","last_became_peered":"2024-04-16T15:40:24.534241+0000","last_unstale":"2024-04-16T15:40:29.918672+0000","last_undegraded":"2024-04-16T15:40:29.918672+0000","last_fullsized":"2024-04-16T15:40:29.918672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:30.065862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567152+0000","last_change":"2024-04-16T15:40:24.546408+0000","last_active":"2024-04-16T15:40:29.567152+0000","last_peered":"2024-04-16T15:40:29.567152+0000","last_clean":"2024-04-16T15:40:29.567152+0000","last_became_active":"2024-04-16T15:40:24.546168+0000","last_became_peered":"2024-04-16T15:40:24.546168+0000","last_unstale":"2024-04-16T15:40:29.567152+0000","last_undegraded":"2024-04-16T15:40:29.567152+0000","last_fullsized":"2024-04-16T15:40:29.567152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920159+0000","last_change":"2024-04-16T15:40:24.533057+0000","last_active":"2024-04-16T15:40:29.920159+0000","last_peered":"2024-04-16T15:40:29.920159+0000","last_clean":"2024-04-16T15:40:29.920159+0000","last_became_active":"2024-04-16T15:40:24.532732+0000","last_became_peered":"2024-04-16T15:40:24.532732+0000","last_unstale":"2024-04-16T15:40:29.920159+0000","last_undegraded":"2024-04-16T15:40:29.920159+0000","last_fullsized":"2024-04-16T15:40:29.920159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:13.766871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567143+0000","last_change":"2024-04-16T15:40:24.540869+0000","last_active":"2024-04-16T15:40:29.567143+0000","last_peered":"2024-04-16T15:40:29.567143+0000","last_clean":"2024-04-16T15:40:29.567143+0000","last_became_active":"2024-04-16T15:40:24.537498+0000","last_became_peered":"2024-04-16T15:40:24.537498+0000","last_unstale":"2024-04-16T15:40:29.567143+0000","last_undegraded":"2024-04-16T15:40:29.567143+0000","last_fullsized":"2024-04-16T15:40:29.567143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567216+0000","last_change":"2024-04-16T15:40:24.536207+0000","last_active":"2024-04-16T15:40:29.567216+0000","last_peered":"2024-04-16T15:40:29.567216+0000","last_clean":"2024-04-16T15:40:29.567216+0000","last_became_active":"2024-04-16T15:40:24.535940+0000","last_became_peered":"2024-04-16T15:40:24.535940+0000","last_unstale":"2024-04-16T15:40:29.567216+0000","last_undegraded":"2024-04-16T15:40:29.567216+0000","last_fullsized":"2024-04-16T15:40:29.567216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920836+0000","last_change":"2024-04-16T15:40:24.544527+0000","last_active":"2024-04-16T15:40:29.920836+0000","last_peered":"2024-04-16T15:40:29.920836+0000","last_clean":"2024-04-16T15:40:29.920836+0000","last_became_active":"2024-04-16T15:40:24.543702+0000","last_became_peered":"2024-04-16T15:40:24.543702+0000","last_unstale":"2024-04-16T15:40:29.920836+0000","last_undegraded":"2024-04-16T15:40:29.920836+0000","last_fullsized":"2024-04-16T15:40:29.920836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:24.380253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566076+0000","last_change":"2024-04-16T15:40:24.545972+0000","last_active":"2024-04-16T15:40:29.566076+0000","last_peered":"2024-04-16T15:40:29.566076+0000","last_clean":"2024-04-16T15:40:29.566076+0000","last_became_active":"2024-04-16T15:40:24.545726+0000","last_became_peered":"2024-04-16T15:40:24.545726+0000","last_unstale":"2024-04-16T15:40:29.566076+0000","last_undegraded":"2024-04-16T15:40:29.566076+0000","last_fullsized":"2024-04-16T15:40:29.566076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919710+0000","last_change":"2024-04-16T15:40:24.550333+0000","last_active":"2024-04-16T15:40:29.919710+0000","last_peered":"2024-04-16T15:40:29.919710+0000","last_clean":"2024-04-16T15:40:29.919710+0000","last_became_active":"2024-04-16T15:40:24.550063+0000","last_became_peered":"2024-04-16T15:40:24.550063+0000","last_unstale":"2024-04-16T15:40:29.919710+0000","last_undegraded":"2024-04-16T15:40:29.919710+0000","last_fullsized":"2024-04-16T15:40:29.919710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920040+0000","last_change":"2024-04-16T15:40:24.545717+0000","last_active":"2024-04-16T15:40:29.920040+0000","last_peered":"2024-04-16T15:40:29.920040+0000","last_clean":"2024-04-16T15:40:29.920040+0000","last_became_active":"2024-04-16T15:40:24.545445+0000","last_became_peered":"2024-04-16T15:40:24.545445+0000","last_unstale":"2024-04-16T15:40:29.920040+0000","last_undegraded":"2024-04-16T15:40:29.920040+0000","last_fullsized":"2024-04-16T15:40:29.920040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919205+0000","last_change":"2024-04-16T15:40:24.549202+0000","last_active":"2024-04-16T15:40:29.919205+0000","last_peered":"2024-04-16T15:40:29.919205+0000","last_clean":"2024-04-16T15:40:29.919205+0000","last_became_active":"2024-04-16T15:40:24.548794+0000","last_became_peered":"2024-04-16T15:40:24.548794+0000","last_unstale":"2024-04-16T15:40:29.919205+0000","last_undegraded":"2024-04-16T15:40:29.919205+0000","last_fullsized":"2024-04-16T15:40:29.919205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:25.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919742+0000","last_change":"2024-04-16T15:40:24.548634+0000","last_active":"2024-04-16T15:40:29.919742+0000","last_peered":"2024-04-16T15:40:29.919742+0000","last_clean":"2024-04-16T15:40:29.919742+0000","last_became_active":"2024-04-16T15:40:24.548342+0000","last_became_peered":"2024-04-16T15:40:24.548342+0000","last_unstale":"2024-04-16T15:40:29.919742+0000","last_undegraded":"2024-04-16T15:40:29.919742+0000","last_fullsized":"2024-04-16T15:40:29.919742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:18.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918832+0000","last_change":"2024-04-16T15:40:24.534160+0000","last_active":"2024-04-16T15:40:29.918832+0000","last_peered":"2024-04-16T15:40:29.918832+0000","last_clean":"2024-04-16T15:40:29.918832+0000","last_became_active":"2024-04-16T15:40:24.533913+0000","last_became_peered":"2024-04-16T15:40:24.533913+0000","last_unstale":"2024-04-16T15:40:29.918832+0000","last_undegraded":"2024-04-16T15:40:29.918832+0000","last_fullsized":"2024-04-16T15:40:29.918832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769869+0000","last_change":"2024-04-16T15:40:24.546471+0000","last_active":"2024-04-16T15:40:29.769869+0000","last_peered":"2024-04-16T15:40:29.769869+0000","last_clean":"2024-04-16T15:40:29.769869+0000","last_became_active":"2024-04-16T15:40:24.545638+0000","last_became_peered":"2024-04-16T15:40:24.545638+0000","last_unstale":"2024-04-16T15:40:29.769869+0000","last_undegraded":"2024-04-16T15:40:29.769869+0000","last_fullsized":"2024-04-16T15:40:29.769869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920784+0000","last_change":"2024-04-16T15:40:24.546172+0000","last_active":"2024-04-16T15:40:29.920784+0000","last_peered":"2024-04-16T15:40:29.920784+0000","last_clean":"2024-04-16T15:40:29.920784+0000","last_became_active":"2024-04-16T15:40:24.545582+0000","last_became_peered":"2024-04-16T15:40:24.545582+0000","last_unstale":"2024-04-16T15:40:29.920784+0000","last_undegraded":"2024-04-16T15:40:29.920784+0000","last_fullsized":"2024-04-16T15:40:29.920784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:21.801823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:40:44.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:40:44.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.603+0000 7f4158ce6640 1 Processor -- start 2024-04-16T15:40:44.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.603+0000 7f4158ce6640 1 -- start start 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4158ce6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f415406cbb0 0x7f4154072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4158ce6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f41540c3f40 0x7f41540c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4158ce6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41540c5b00 con 0x7f41540c2fa0 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4158ce6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f41540c6390 con 0x7f41540c3f40 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4158ce6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f41540c6670 con 0x7f415406cbb0 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f415259c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f415406cbb0 0x7f4154072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4151d9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f41540c3f40 0x7f41540c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4152d9d640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f41540c2fa0 0x7f41540c0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:60774/0 (socket says 172.21.15.71:60774) 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f415259c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f415406cbb0 0x7f4154072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:34392/0 (socket says 172.21.15.71:34392) 2024-04-16T15:40:44.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4152d9d640 1 -- 172.21.15.71:0/2016211582 learned_addr learned my addr 172.21.15.71:0/2016211582 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4151d9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f41540c3f40 0x7f41540c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:51308/0 (socket says 172.21.15.71:51308) 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f415259c640 1 -- 172.21.15.71:0/2016211582 >> v2:172.21.15.122:3300/0 conn(0x7f41540c3f40 msgr2=0x7f41540c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f415259c640 1 --2- 172.21.15.71:0/2016211582 >> v2:172.21.15.122:3300/0 conn(0x7f41540c3f40 0x7f41540c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f415259c640 1 -- 172.21.15.71:0/2016211582 >> v1:172.21.15.122:6789/0 conn(0x7f41540c2fa0 legacy=0x7f41540c0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f415259c640 1 -- 172.21.15.71:0/2016211582 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41540c5da0 con 0x7f415406cbb0 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.604+0000 7f4151d9b640 1 --2- 172.21.15.71:0/2016211582 >> v2:172.21.15.122:3300/0 conn(0x7f41540c3f40 0x7f41540c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.605+0000 7f415259c640 1 --2- 172.21.15.71:0/2016211582 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f415406cbb0 0x7f4154072240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f41440097a0 tx=0x7f414402b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54df26b6b987073 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.605+0000 7f415159a640 1 -- 172.21.15.71:0/2016211582 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144038070 con 0x7f415406cbb0 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.605+0000 7f415159a640 1 -- 172.21.15.71:0/2016211582 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4144031700 con 0x7f415406cbb0 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.605+0000 7f415159a640 1 -- 172.21.15.71:0/2016211582 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144031ad0 con 0x7f415406cbb0 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.606+0000 7f4158ce6640 1 -- 172.21.15.71:0/2016211582 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f415406cbb0 msgr2=0x7f4154072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.606+0000 7f4158ce6640 1 --2- 172.21.15.71:0/2016211582 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f415406cbb0 0x7f4154072240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f41440097a0 tx=0x7f414402b7a0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.606+0000 7f4158ce6640 1 -- 172.21.15.71:0/2016211582 shutdown_connections 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.606+0000 7f4158ce6640 1 --2- 172.21.15.71:0/2016211582 >> v2:172.21.15.122:3300/0 conn(0x7f41540c3f40 0x7f41540c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.606+0000 7f4158ce6640 1 --2- 172.21.15.71:0/2016211582 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f415406cbb0 0x7f4154072240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.606+0000 7f4158ce6640 1 -- 172.21.15.71:0/2016211582 >> 172.21.15.71:0/2016211582 conn(0x7f41540c45d0 msgr2=0x7f41540c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.606+0000 7f4158ce6640 1 -- 172.21.15.71:0/2016211582 shutdown_connections 2024-04-16T15:40:44.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.606+0000 7f4158ce6640 1 -- 172.21.15.71:0/2016211582 wait complete. 2024-04-16T15:40:44.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.607+0000 7f4158ce6640 1 Processor -- start 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.607+0000 7f4158ce6640 1 -- start start 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4158ce6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f415406cbb0 0x7f41540a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4158ce6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41540c2fa0 0x7f41541eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4158ce6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41540c3f40 0x7f41541ebd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4158ce6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f415406cf90 con 0x7f41540c2fa0 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4158ce6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4154073060 con 0x7f415406cbb0 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4158ce6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f415406d110 con 0x7f41540c3f40 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4151d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41540c2fa0 0x7f41541eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4151d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41540c2fa0 0x7f41541eb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47072/0 (socket says 172.21.15.71:47072) 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4151d9b640 1 -- 172.21.15.71:0/1639929529 learned_addr learned my addr 172.21.15.71:0/1639929529 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:44.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f415259c640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f415406cbb0 0x7f41540a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.608+0000 7f4152d9d640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41540c3f40 0x7f41541ebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4151d9b640 1 -- 172.21.15.71:0/1639929529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41540c3f40 msgr2=0x7f41541ebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4151d9b640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41540c3f40 0x7f41541ebd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4151d9b640 1 -- 172.21.15.71:0/1639929529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f415406cbb0 msgr2=0x7f41540a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4151d9b640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f415406cbb0 0x7f41540a7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4151d9b640 1 -- 172.21.15.71:0/1639929529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41541ef630 con 0x7f41540c2fa0 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4151d9b640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41540c2fa0 0x7f41541eb7f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f414800ebe0 tx=0x7f414800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4142ffd640 1 -- 172.21.15.71:0/1639929529 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148017070 con 0x7f41540c2fa0 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4142ffd640 1 -- 172.21.15.71:0/1639929529 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414800cbd0 con 0x7f41540c2fa0 2024-04-16T15:40:44.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4142ffd640 1 -- 172.21.15.71:0/1639929529 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148013470 con 0x7f41540c2fa0 2024-04-16T15:40:44.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41541e0ae0 con 0x7f41540c2fa0 2024-04-16T15:40:44.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.609+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41541eb2d0 con 0x7f41540c2fa0 2024-04-16T15:40:44.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.610+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41541eb0c0 con 0x7f41540c2fa0 2024-04-16T15:40:44.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.610+0000 7f4142ffd640 1 -- 172.21.15.71:0/1639929529 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f414801a070 con 0x7f41540c2fa0 2024-04-16T15:40:44.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.611+0000 7f4142ffd640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f413406c8f0 0x7f413406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.611+0000 7f4142ffd640 1 -- 172.21.15.71:0/1639929529 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f414808bc40 con 0x7f41540c2fa0 2024-04-16T15:40:44.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.611+0000 7f415259c640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f413406c8f0 0x7f413406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.611+0000 7f415259c640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f413406c8f0 0x7f413406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4144035000 tx=0x7f4144008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.615+0000 7f4142ffd640 1 -- 172.21.15.71:0/1639929529 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4148059770 con 0x7f41540c2fa0 2024-04-16T15:40:44.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.786+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4154068d20 con 0x7f413406c8f0 2024-04-16T15:40:44.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.796+0000 7f4142ffd640 1 -- 172.21.15.71:0/1639929529 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f4154068d20 con 0x7f413406c8f0 2024-04-16T15:40:44.800 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:44.802 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:40:44.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f413406c8f0 msgr2=0x7f413406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f413406c8f0 0x7f413406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4144035000 tx=0x7f4144008030 comp rx=0 tx=0).stop 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41540c2fa0 msgr2=0x7f41541eb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41540c2fa0 0x7f41541eb7f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f414800ebe0 tx=0x7f414800c520 comp rx=0 tx=0).stop 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 shutdown_connections 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f413406c8f0 0x7f413406edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41540c3f40 0x7f41541ebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41540c2fa0 0x7f41541eb7f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 --2- 172.21.15.71:0/1639929529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f415406cbb0 0x7f41540a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 >> 172.21.15.71:0/1639929529 conn(0x7f41540c45d0 msgr2=0x7f41541da830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 shutdown_connections 2024-04-16T15:40:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.799+0000 7f4158ce6640 1 -- 172.21.15.71:0/1639929529 wait complete. 2024-04-16T15:40:44.811 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-16T15:40:43.699345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219396,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755324,"statfs":{"total":773094113280,"available":772869451776,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002569"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919216+0000","last_change":"2024-04-16T15:40:23.532452+0000","last_active":"2024-04-16T15:40:29.919216+0000","last_peered":"2024-04-16T15:40:29.919216+0000","last_clean":"2024-04-16T15:40:29.919216+0000","last_became_active":"2024-04-16T15:40:23.531709+0000","last_became_peered":"2024-04-16T15:40:23.531709+0000","last_unstale":"2024-04-16T15:40:29.919216+0000","last_undegraded":"2024-04-16T15:40:29.919216+0000","last_fullsized":"2024-04-16T15:40:29.919216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:52.586318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567735+0000","last_change":"2024-04-16T15:40:24.542948+0000","last_active":"2024-04-16T15:40:29.567735+0000","last_peered":"2024-04-16T15:40:29.567735+0000","last_clean":"2024-04-16T15:40:29.567735+0000","last_became_active":"2024-04-16T15:40:24.537898+0000","last_became_peered":"2024-04-16T15:40:24.537898+0000","last_unstale":"2024-04-16T15:40:29.567735+0000","last_undegraded":"2024-04-16T15:40:29.567735+0000","last_fullsized":"2024-04-16T15:40:29.567735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:18.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566318+0000","last_change":"2024-04-16T15:40:28.807073+0000","last_active":"2024-04-16T15:40:29.566318+0000","last_peered":"2024-04-16T15:40:29.566318+0000","last_clean":"2024-04-16T15:40:29.566318+0000","last_became_active":"2024-04-16T15:40:28.806446+0000","last_became_peered":"2024-04-16T15:40:28.806446+0000","last_unstale":"2024-04-16T15:40:29.566318+0000","last_undegraded":"2024-04-16T15:40:29.566318+0000","last_fullsized":"2024-04-16T15:40:29.566318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:54.213728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567940+0000","last_change":"2024-04-16T15:40:23.526932+0000","last_active":"2024-04-16T15:40:29.567940+0000","last_peered":"2024-04-16T15:40:29.567940+0000","last_clean":"2024-04-16T15:40:29.567940+0000","last_became_active":"2024-04-16T15:40:23.526663+0000","last_became_peered":"2024-04-16T15:40:23.526663+0000","last_unstale":"2024-04-16T15:40:29.567940+0000","last_undegraded":"2024-04-16T15:40:29.567940+0000","last_fullsized":"2024-04-16T15:40:29.567940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:52.586318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919103+0000","last_change":"2024-04-16T15:40:24.541524+0000","last_active":"2024-04-16T15:40:29.919103+0000","last_peered":"2024-04-16T15:40:29.919103+0000","last_clean":"2024-04-16T15:40:29.919103+0000","last_became_active":"2024-04-16T15:40:24.540243+0000","last_became_peered":"2024-04-16T15:40:24.540243+0000","last_unstale":"2024-04-16T15:40:29.919103+0000","last_undegraded":"2024-04-16T15:40:29.919103+0000","last_fullsized":"2024-04-16T15:40:29.919103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919188+0000","last_change":"2024-04-16T15:40:28.569991+0000","last_active":"2024-04-16T15:40:29.919188+0000","last_peered":"2024-04-16T15:40:29.919188+0000","last_clean":"2024-04-16T15:40:29.919188+0000","last_became_active":"2024-04-16T15:40:28.569596+0000","last_became_peered":"2024-04-16T15:40:28.569596+0000","last_unstale":"2024-04-16T15:40:29.919188+0000","last_undegraded":"2024-04-16T15:40:29.919188+0000","last_fullsized":"2024-04-16T15:40:29.919188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:21.999059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921685+0000","last_change":"2024-04-16T15:40:23.535294+0000","last_active":"2024-04-16T15:40:29.921685+0000","last_peered":"2024-04-16T15:40:29.921685+0000","last_clean":"2024-04-16T15:40:29.921685+0000","last_became_active":"2024-04-16T15:40:23.534958+0000","last_became_peered":"2024-04-16T15:40:23.534958+0000","last_unstale":"2024-04-16T15:40:29.921685+0000","last_undegraded":"2024-04-16T15:40:29.921685+0000","last_fullsized":"2024-04-16T15:40:29.921685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:34:59.914744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919928+0000","last_change":"2024-04-16T15:40:24.544556+0000","last_active":"2024-04-16T15:40:29.919928+0000","last_peered":"2024-04-16T15:40:29.919928+0000","last_clean":"2024-04-16T15:40:29.919928+0000","last_became_active":"2024-04-16T15:40:24.544072+0000","last_became_peered":"2024-04-16T15:40:24.544072+0000","last_unstale":"2024-04-16T15:40:29.919928+0000","last_undegraded":"2024-04-16T15:40:29.919928+0000","last_fullsized":"2024-04-16T15:40:29.919928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:25.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920842+0000","last_change":"2024-04-16T15:40:28.570724+0000","last_active":"2024-04-16T15:40:29.920842+0000","last_peered":"2024-04-16T15:40:29.920842+0000","last_clean":"2024-04-16T15:40:29.920842+0000","last_became_active":"2024-04-16T15:40:28.570338+0000","last_became_peered":"2024-04-16T15:40:28.570338+0000","last_unstale":"2024-04-16T15:40:29.920842+0000","last_undegraded":"2024-04-16T15:40:29.920842+0000","last_fullsized":"2024-04-16T15:40:29.920842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:55.765984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920279+0000","last_change":"2024-04-16T15:40:23.536319+0000","last_active":"2024-04-16T15:40:29.920279+0000","last_peered":"2024-04-16T15:40:29.920279+0000","last_clean":"2024-04-16T15:40:29.920279+0000","last_became_active":"2024-04-16T15:40:23.536059+0000","last_became_peered":"2024-04-16T15:40:23.536059+0000","last_unstale":"2024-04-16T15:40:29.920279+0000","last_undegraded":"2024-04-16T15:40:29.920279+0000","last_fullsized":"2024-04-16T15:40:29.920279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:19.974713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920444+0000","last_change":"2024-04-16T15:40:24.543661+0000","last_active":"2024-04-16T15:40:29.920444+0000","last_peered":"2024-04-16T15:40:29.920444+0000","last_clean":"2024-04-16T15:40:29.920444+0000","last_became_active":"2024-04-16T15:40:24.543414+0000","last_became_peered":"2024-04-16T15:40:24.543414+0000","last_unstale":"2024-04-16T15:40:29.920444+0000","last_undegraded":"2024-04-16T15:40:29.920444+0000","last_fullsized":"2024-04-16T15:40:29.920444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:38.664081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919473+0000","last_change":"2024-04-16T15:40:28.566254+0000","last_active":"2024-04-16T15:40:29.919473+0000","last_peered":"2024-04-16T15:40:29.919473+0000","last_clean":"2024-04-16T15:40:29.919473+0000","last_became_active":"2024-04-16T15:40:28.565804+0000","last_became_peered":"2024-04-16T15:40:28.565804+0000","last_unstale":"2024-04-16T15:40:29.919473+0000","last_undegraded":"2024-04-16T15:40:29.919473+0000","last_fullsized":"2024-04-16T15:40:29.919473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:40.008247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566632+0000","last_change":"2024-04-16T15:40:24.539072+0000","last_active":"2024-04-16T15:40:29.566632+0000","last_peered":"2024-04-16T15:40:29.566632+0000","last_clean":"2024-04-16T15:40:29.566632+0000","last_became_active":"2024-04-16T15:40:24.537100+0000","last_became_peered":"2024-04-16T15:40:24.537100+0000","last_unstale":"2024-04-16T15:40:29.566632+0000","last_undegraded":"2024-04-16T15:40:29.566632+0000","last_fullsized":"2024-04-16T15:40:29.566632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920246+0000","last_change":"2024-04-16T15:40:23.537232+0000","last_active":"2024-04-16T15:40:29.920246+0000","last_peered":"2024-04-16T15:40:29.920246+0000","last_clean":"2024-04-16T15:40:29.920246+0000","last_became_active":"2024-04-16T15:40:23.536233+0000","last_became_peered":"2024-04-16T15:40:23.536233+0000","last_unstale":"2024-04-16T15:40:29.920246+0000","last_undegraded":"2024-04-16T15:40:29.920246+0000","last_fullsized":"2024-04-16T15:40:29.920246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:43.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921321+0000","last_change":"2024-04-16T15:40:28.570768+0000","last_active":"2024-04-16T15:40:29.921321+0000","last_peered":"2024-04-16T15:40:29.921321+0000","last_clean":"2024-04-16T15:40:29.921321+0000","last_became_active":"2024-04-16T15:40:28.570416+0000","last_became_peered":"2024-04-16T15:40:28.570416+0000","last_unstale":"2024-04-16T15:40:29.921321+0000","last_undegraded":"2024-04-16T15:40:29.921321+0000","last_fullsized":"2024-04-16T15:40:29.921321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567237+0000","last_change":"2024-04-16T15:40:24.548809+0000","last_active":"2024-04-16T15:40:29.567237+0000","last_peered":"2024-04-16T15:40:29.567237+0000","last_clean":"2024-04-16T15:40:29.567237+0000","last_became_active":"2024-04-16T15:40:24.548463+0000","last_became_peered":"2024-04-16T15:40:24.548463+0000","last_unstale":"2024-04-16T15:40:29.567237+0000","last_undegraded":"2024-04-16T15:40:29.567237+0000","last_fullsized":"2024-04-16T15:40:29.567237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769455+0000","last_change":"2024-04-16T15:40:23.531263+0000","last_active":"2024-04-16T15:40:29.769455+0000","last_peered":"2024-04-16T15:40:29.769455+0000","last_clean":"2024-04-16T15:40:29.769455+0000","last_became_active":"2024-04-16T15:40:23.530998+0000","last_became_peered":"2024-04-16T15:40:23.530998+0000","last_unstale":"2024-04-16T15:40:29.769455+0000","last_undegraded":"2024-04-16T15:40:29.769455+0000","last_fullsized":"2024-04-16T15:40:29.769455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919978+0000","last_change":"2024-04-16T15:40:28.569962+0000","last_active":"2024-04-16T15:40:29.919978+0000","last_peered":"2024-04-16T15:40:29.919978+0000","last_clean":"2024-04-16T15:40:29.919978+0000","last_became_active":"2024-04-16T15:40:28.569572+0000","last_became_peered":"2024-04-16T15:40:28.569572+0000","last_unstale":"2024-04-16T15:40:29.919978+0000","last_undegraded":"2024-04-16T15:40:29.919978+0000","last_fullsized":"2024-04-16T15:40:29.919978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:18.103136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566305+0000","last_change":"2024-04-16T15:40:24.544276+0000","last_active":"2024-04-16T15:40:29.566305+0000","last_peered":"2024-04-16T15:40:29.566305+0000","last_clean":"2024-04-16T15:40:29.566305+0000","last_became_active":"2024-04-16T15:40:24.544035+0000","last_became_peered":"2024-04-16T15:40:24.544035+0000","last_unstale":"2024-04-16T15:40:29.566305+0000","last_undegraded":"2024-04-16T15:40:29.566305+0000","last_fullsized":"2024-04-16T15:40:29.566305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769209+0000","last_change":"2024-04-16T15:40:23.530210+0000","last_active":"2024-04-16T15:40:29.769209+0000","last_peered":"2024-04-16T15:40:29.769209+0000","last_clean":"2024-04-16T15:40:29.769209+0000","last_became_active":"2024-04-16T15:40:23.529955+0000","last_became_peered":"2024-04-16T15:40:23.529955+0000","last_unstale":"2024-04-16T15:40:29.769209+0000","last_undegraded":"2024-04-16T15:40:29.769209+0000","last_fullsized":"2024-04-16T15:40:29.769209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:21.943427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769997+0000","last_change":"2024-04-16T15:40:28.805304+0000","last_active":"2024-04-16T15:40:29.769997+0000","last_peered":"2024-04-16T15:40:29.769997+0000","last_clean":"2024-04-16T15:40:29.769997+0000","last_became_active":"2024-04-16T15:40:28.804926+0000","last_became_peered":"2024-04-16T15:40:28.804926+0000","last_unstale":"2024-04-16T15:40:29.769997+0000","last_undegraded":"2024-04-16T15:40:29.769997+0000","last_fullsized":"2024-04-16T15:40:29.769997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:25.828695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919801+0000","last_change":"2024-04-16T15:40:24.537825+0000","last_active":"2024-04-16T15:40:29.919801+0000","last_peered":"2024-04-16T15:40:29.919801+0000","last_clean":"2024-04-16T15:40:29.919801+0000","last_became_active":"2024-04-16T15:40:24.536478+0000","last_became_peered":"2024-04-16T15:40:24.536478+0000","last_unstale":"2024-04-16T15:40:29.919801+0000","last_undegraded":"2024-04-16T15:40:29.919801+0000","last_fullsized":"2024-04-16T15:40:29.919801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:26.549813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919917+0000","last_change":"2024-04-16T15:40:23.535794+0000","last_active":"2024-04-16T15:40:29.919917+0000","last_peered":"2024-04-16T15:40:29.919917+0000","last_clean":"2024-04-16T15:40:29.919917+0000","last_became_active":"2024-04-16T15:40:23.535373+0000","last_became_peered":"2024-04-16T15:40:23.535373+0000","last_unstale":"2024-04-16T15:40:29.919917+0000","last_undegraded":"2024-04-16T15:40:29.919917+0000","last_fullsized":"2024-04-16T15:40:29.919917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920113+0000","last_change":"2024-04-16T15:40:28.561348+0000","last_active":"2024-04-16T15:40:29.920113+0000","last_peered":"2024-04-16T15:40:29.920113+0000","last_clean":"2024-04-16T15:40:29.920113+0000","last_became_active":"2024-04-16T15:40:28.560833+0000","last_became_peered":"2024-04-16T15:40:28.560833+0000","last_unstale":"2024-04-16T15:40:29.920113+0000","last_undegraded":"2024-04-16T15:40:29.920113+0000","last_fullsized":"2024-04-16T15:40:29.920113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:22.352368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919131+0000","last_change":"2024-04-16T15:40:24.534724+0000","last_active":"2024-04-16T15:40:29.919131+0000","last_peered":"2024-04-16T15:40:29.919131+0000","last_clean":"2024-04-16T15:40:29.919131+0000","last_became_active":"2024-04-16T15:40:24.534469+0000","last_became_peered":"2024-04-16T15:40:24.534469+0000","last_unstale":"2024-04-16T15:40:29.919131+0000","last_undegraded":"2024-04-16T15:40:29.919131+0000","last_fullsized":"2024-04-16T15:40:29.919131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:50:03.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567368+0000","last_change":"2024-04-16T15:40:23.531841+0000","last_active":"2024-04-16T15:40:29.567368+0000","last_peered":"2024-04-16T15:40:29.567368+0000","last_clean":"2024-04-16T15:40:29.567368+0000","last_became_active":"2024-04-16T15:40:23.531565+0000","last_became_peered":"2024-04-16T15:40:23.531565+0000","last_unstale":"2024-04-16T15:40:29.567368+0000","last_undegraded":"2024-04-16T15:40:29.567368+0000","last_fullsized":"2024-04-16T15:40:29.567368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919600+0000","last_change":"2024-04-16T15:40:28.807348+0000","last_active":"2024-04-16T15:40:29.919600+0000","last_peered":"2024-04-16T15:40:29.919600+0000","last_clean":"2024-04-16T15:40:29.919600+0000","last_became_active":"2024-04-16T15:40:28.806984+0000","last_became_peered":"2024-04-16T15:40:28.806984+0000","last_unstale":"2024-04-16T15:40:29.919600+0000","last_undegraded":"2024-04-16T15:40:29.919600+0000","last_fullsized":"2024-04-16T15:40:29.919600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:30.543430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567817+0000","last_change":"2024-04-16T15:40:24.544401+0000","last_active":"2024-04-16T15:40:29.567817+0000","last_peered":"2024-04-16T15:40:29.567817+0000","last_clean":"2024-04-16T15:40:29.567817+0000","last_became_active":"2024-04-16T15:40:24.544156+0000","last_became_peered":"2024-04-16T15:40:24.544156+0000","last_unstale":"2024-04-16T15:40:29.567817+0000","last_undegraded":"2024-04-16T15:40:29.567817+0000","last_fullsized":"2024-04-16T15:40:29.567817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769950+0000","last_change":"2024-04-16T15:40:23.532974+0000","last_active":"2024-04-16T15:40:29.769950+0000","last_peered":"2024-04-16T15:40:29.769950+0000","last_clean":"2024-04-16T15:40:29.769950+0000","last_became_active":"2024-04-16T15:40:23.532692+0000","last_became_peered":"2024-04-16T15:40:23.532692+0000","last_unstale":"2024-04-16T15:40:29.769950+0000","last_undegraded":"2024-04-16T15:40:29.769950+0000","last_fullsized":"2024-04-16T15:40:29.769950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:19.974713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566451+0000","last_change":"2024-04-16T15:40:28.806764+0000","last_active":"2024-04-16T15:40:29.566451+0000","last_peered":"2024-04-16T15:40:29.566451+0000","last_clean":"2024-04-16T15:40:29.566451+0000","last_became_active":"2024-04-16T15:40:28.806437+0000","last_became_peered":"2024-04-16T15:40:28.806437+0000","last_unstale":"2024-04-16T15:40:29.566451+0000","last_undegraded":"2024-04-16T15:40:29.566451+0000","last_fullsized":"2024-04-16T15:40:29.566451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:52:28.865900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769595+0000","last_change":"2024-04-16T15:40:24.536656+0000","last_active":"2024-04-16T15:40:29.769595+0000","last_peered":"2024-04-16T15:40:29.769595+0000","last_clean":"2024-04-16T15:40:29.769595+0000","last_became_active":"2024-04-16T15:40:24.536405+0000","last_became_peered":"2024-04-16T15:40:24.536405+0000","last_unstale":"2024-04-16T15:40:29.769595+0000","last_undegraded":"2024-04-16T15:40:29.769595+0000","last_fullsized":"2024-04-16T15:40:29.769595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921784+0000","last_change":"2024-04-16T15:40:23.536026+0000","last_active":"2024-04-16T15:40:29.921784+0000","last_peered":"2024-04-16T15:40:29.921784+0000","last_clean":"2024-04-16T15:40:29.921784+0000","last_became_active":"2024-04-16T15:40:23.535220+0000","last_became_peered":"2024-04-16T15:40:23.535220+0000","last_unstale":"2024-04-16T15:40:29.921784+0000","last_undegraded":"2024-04-16T15:40:29.921784+0000","last_fullsized":"2024-04-16T15:40:29.921784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:37.570307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918806+0000","last_change":"2024-04-16T15:40:28.804863+0000","last_active":"2024-04-16T15:40:29.918806+0000","last_peered":"2024-04-16T15:40:29.918806+0000","last_clean":"2024-04-16T15:40:29.918806+0000","last_became_active":"2024-04-16T15:40:28.804502+0000","last_became_peered":"2024-04-16T15:40:28.804502+0000","last_unstale":"2024-04-16T15:40:29.918806+0000","last_undegraded":"2024-04-16T15:40:29.918806+0000","last_fullsized":"2024-04-16T15:40:29.918806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920557+0000","last_change":"2024-04-16T15:40:24.551868+0000","last_active":"2024-04-16T15:40:29.920557+0000","last_peered":"2024-04-16T15:40:29.920557+0000","last_clean":"2024-04-16T15:40:29.920557+0000","last_became_active":"2024-04-16T15:40:24.550950+0000","last_became_peered":"2024-04-16T15:40:24.550950+0000","last_unstale":"2024-04-16T15:40:29.920557+0000","last_undegraded":"2024-04-16T15:40:29.920557+0000","last_fullsized":"2024-04-16T15:40:29.920557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:27.826954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769685+0000","last_change":"2024-04-16T15:40:23.533782+0000","last_active":"2024-04-16T15:40:29.769685+0000","last_peered":"2024-04-16T15:40:29.769685+0000","last_clean":"2024-04-16T15:40:29.769685+0000","last_became_active":"2024-04-16T15:40:23.533547+0000","last_became_peered":"2024-04-16T15:40:23.533547+0000","last_unstale":"2024-04-16T15:40:29.769685+0000","last_undegraded":"2024-04-16T15:40:29.769685+0000","last_fullsized":"2024-04-16T15:40:29.769685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769337+0000","last_change":"2024-04-16T15:40:28.570047+0000","last_active":"2024-04-16T15:40:29.769337+0000","last_peered":"2024-04-16T15:40:29.769337+0000","last_clean":"2024-04-16T15:40:29.769337+0000","last_became_active":"2024-04-16T15:40:28.569688+0000","last_became_peered":"2024-04-16T15:40:28.569688+0000","last_unstale":"2024-04-16T15:40:29.769337+0000","last_undegraded":"2024-04-16T15:40:29.769337+0000","last_fullsized":"2024-04-16T15:40:29.769337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:42.690952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921046+0000","last_change":"2024-04-16T15:40:24.549077+0000","last_active":"2024-04-16T15:40:29.921046+0000","last_peered":"2024-04-16T15:40:29.921046+0000","last_clean":"2024-04-16T15:40:29.921046+0000","last_became_active":"2024-04-16T15:40:24.548802+0000","last_became_peered":"2024-04-16T15:40:24.548802+0000","last_unstale":"2024-04-16T15:40:29.921046+0000","last_undegraded":"2024-04-16T15:40:29.921046+0000","last_fullsized":"2024-04-16T15:40:29.921046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:50.142130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920546+0000","last_change":"2024-04-16T15:40:23.528707+0000","last_active":"2024-04-16T15:40:29.920546+0000","last_peered":"2024-04-16T15:40:29.920546+0000","last_clean":"2024-04-16T15:40:29.920546+0000","last_became_active":"2024-04-16T15:40:23.528426+0000","last_became_peered":"2024-04-16T15:40:23.528426+0000","last_unstale":"2024-04-16T15:40:29.920546+0000","last_undegraded":"2024-04-16T15:40:29.920546+0000","last_fullsized":"2024-04-16T15:40:29.920546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:12.762446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769721+0000","last_change":"2024-04-16T15:40:28.561110+0000","last_active":"2024-04-16T15:40:29.769721+0000","last_peered":"2024-04-16T15:40:29.769721+0000","last_clean":"2024-04-16T15:40:29.769721+0000","last_became_active":"2024-04-16T15:40:28.560719+0000","last_became_peered":"2024-04-16T15:40:28.560719+0000","last_unstale":"2024-04-16T15:40:29.769721+0000","last_undegraded":"2024-04-16T15:40:29.769721+0000","last_fullsized":"2024-04-16T15:40:29.769721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:31.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769261+0000","last_change":"2024-04-16T15:40:24.536757+0000","last_active":"2024-04-16T15:40:29.769261+0000","last_peered":"2024-04-16T15:40:29.769261+0000","last_clean":"2024-04-16T15:40:29.769261+0000","last_became_active":"2024-04-16T15:40:24.536520+0000","last_became_peered":"2024-04-16T15:40:24.536520+0000","last_unstale":"2024-04-16T15:40:29.769261+0000","last_undegraded":"2024-04-16T15:40:29.769261+0000","last_fullsized":"2024-04-16T15:40:29.769261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566917+0000","last_change":"2024-04-16T15:40:23.520551+0000","last_active":"2024-04-16T15:40:29.566917+0000","last_peered":"2024-04-16T15:40:29.566917+0000","last_clean":"2024-04-16T15:40:29.566917+0000","last_became_active":"2024-04-16T15:40:23.520300+0000","last_became_peered":"2024-04-16T15:40:23.520300+0000","last_unstale":"2024-04-16T15:40:29.566917+0000","last_undegraded":"2024-04-16T15:40:29.566917+0000","last_fullsized":"2024-04-16T15:40:29.566917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.770097+0000","last_change":"2024-04-16T15:40:28.570553+0000","last_active":"2024-04-16T15:40:29.770097+0000","last_peered":"2024-04-16T15:40:29.770097+0000","last_clean":"2024-04-16T15:40:29.770097+0000","last_became_active":"2024-04-16T15:40:28.569635+0000","last_became_peered":"2024-04-16T15:40:28.569635+0000","last_unstale":"2024-04-16T15:40:29.770097+0000","last_undegraded":"2024-04-16T15:40:29.770097+0000","last_fullsized":"2024-04-16T15:40:29.770097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:28.407124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918519+0000","last_change":"2024-04-16T15:40:24.537795+0000","last_active":"2024-04-16T15:40:29.918519+0000","last_peered":"2024-04-16T15:40:29.918519+0000","last_clean":"2024-04-16T15:40:29.918519+0000","last_became_active":"2024-04-16T15:40:24.537528+0000","last_became_peered":"2024-04-16T15:40:24.537528+0000","last_unstale":"2024-04-16T15:40:29.918519+0000","last_undegraded":"2024-04-16T15:40:29.918519+0000","last_fullsized":"2024-04-16T15:40:29.918519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:19:38.574731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920406+0000","last_change":"2024-04-16T15:40:23.533872+0000","last_active":"2024-04-16T15:40:29.920406+0000","last_peered":"2024-04-16T15:40:29.920406+0000","last_clean":"2024-04-16T15:40:29.920406+0000","last_became_active":"2024-04-16T15:40:23.533491+0000","last_became_peered":"2024-04-16T15:40:23.533491+0000","last_unstale":"2024-04-16T15:40:29.920406+0000","last_undegraded":"2024-04-16T15:40:29.920406+0000","last_fullsized":"2024-04-16T15:40:29.920406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:21.943427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919078+0000","last_change":"2024-04-16T15:40:28.807602+0000","last_active":"2024-04-16T15:40:29.919078+0000","last_peered":"2024-04-16T15:40:29.919078+0000","last_clean":"2024-04-16T15:40:29.919078+0000","last_became_active":"2024-04-16T15:40:28.807193+0000","last_became_peered":"2024-04-16T15:40:28.807193+0000","last_unstale":"2024-04-16T15:40:29.919078+0000","last_undegraded":"2024-04-16T15:40:29.919078+0000","last_fullsized":"2024-04-16T15:40:29.919078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:35.057588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567071+0000","last_change":"2024-04-16T15:40:24.546548+0000","last_active":"2024-04-16T15:40:29.567071+0000","last_peered":"2024-04-16T15:40:29.567071+0000","last_clean":"2024-04-16T15:40:29.567071+0000","last_became_active":"2024-04-16T15:40:24.546317+0000","last_became_peered":"2024-04-16T15:40:24.546317+0000","last_unstale":"2024-04-16T15:40:29.567071+0000","last_undegraded":"2024-04-16T15:40:29.567071+0000","last_fullsized":"2024-04-16T15:40:29.567071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:14.076265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921124+0000","last_change":"2024-04-16T15:40:23.529866+0000","last_active":"2024-04-16T15:40:29.921124+0000","last_peered":"2024-04-16T15:40:29.921124+0000","last_clean":"2024-04-16T15:40:29.921124+0000","last_became_active":"2024-04-16T15:40:23.529575+0000","last_became_peered":"2024-04-16T15:40:23.529575+0000","last_unstale":"2024-04-16T15:40:29.921124+0000","last_undegraded":"2024-04-16T15:40:29.921124+0000","last_fullsized":"2024-04-16T15:40:29.921124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920486+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:40:29.920486+0000","last_peered":"2024-04-16T15:40:29.920486+0000","last_clean":"2024-04-16T15:40:29.920486+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:40:29.920486+0000","last_undegraded":"2024-04-16T15:40:29.920486+0000","last_fullsized":"2024-04-16T15:40:29.920486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567180+0000","last_change":"2024-04-16T15:40:28.805311+0000","last_active":"2024-04-16T15:40:29.567180+0000","last_peered":"2024-04-16T15:40:29.567180+0000","last_clean":"2024-04-16T15:40:29.567180+0000","last_became_active":"2024-04-16T15:40:28.804868+0000","last_became_peered":"2024-04-16T15:40:28.804868+0000","last_unstale":"2024-04-16T15:40:29.567180+0000","last_undegraded":"2024-04-16T15:40:29.567180+0000","last_fullsized":"2024-04-16T15:40:29.567180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:32.196316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921580+0000","last_change":"2024-04-16T15:40:24.544114+0000","last_active":"2024-04-16T15:40:29.921580+0000","last_peered":"2024-04-16T15:40:29.921580+0000","last_clean":"2024-04-16T15:40:29.921580+0000","last_became_active":"2024-04-16T15:40:24.543798+0000","last_became_peered":"2024-04-16T15:40:24.543798+0000","last_unstale":"2024-04-16T15:40:29.921580+0000","last_undegraded":"2024-04-16T15:40:29.921580+0000","last_fullsized":"2024-04-16T15:40:29.921580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:38.664081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567401+0000","last_change":"2024-04-16T15:40:23.525608+0000","last_active":"2024-04-16T15:40:29.567401+0000","last_peered":"2024-04-16T15:40:29.567401+0000","last_clean":"2024-04-16T15:40:29.567401+0000","last_became_active":"2024-04-16T15:40:23.525361+0000","last_became_peered":"2024-04-16T15:40:23.525361+0000","last_unstale":"2024-04-16T15:40:29.567401+0000","last_undegraded":"2024-04-16T15:40:29.567401+0000","last_fullsized":"2024-04-16T15:40:29.567401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:17.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566561+0000","last_change":"2024-04-16T15:40:28.806076+0000","last_active":"2024-04-16T15:40:29.566561+0000","last_peered":"2024-04-16T15:40:29.566561+0000","last_clean":"2024-04-16T15:40:29.566561+0000","last_became_active":"2024-04-16T15:40:28.805387+0000","last_became_peered":"2024-04-16T15:40:28.805387+0000","last_unstale":"2024-04-16T15:40:29.566561+0000","last_undegraded":"2024-04-16T15:40:29.566561+0000","last_fullsized":"2024-04-16T15:40:29.566561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:55.765984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920030+0000","last_change":"2024-04-16T15:40:24.546430+0000","last_active":"2024-04-16T15:40:29.920030+0000","last_peered":"2024-04-16T15:40:29.920030+0000","last_clean":"2024-04-16T15:40:29.920030+0000","last_became_active":"2024-04-16T15:40:24.546177+0000","last_became_peered":"2024-04-16T15:40:24.546177+0000","last_unstale":"2024-04-16T15:40:29.920030+0000","last_undegraded":"2024-04-16T15:40:29.920030+0000","last_fullsized":"2024-04-16T15:40:29.920030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:00.952582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769778+0000","last_change":"2024-04-16T15:40:23.537830+0000","last_active":"2024-04-16T15:40:29.769778+0000","last_peered":"2024-04-16T15:40:29.769778+0000","last_clean":"2024-04-16T15:40:29.769778+0000","last_became_active":"2024-04-16T15:40:23.537574+0000","last_became_peered":"2024-04-16T15:40:23.537574+0000","last_unstale":"2024-04-16T15:40:29.769778+0000","last_undegraded":"2024-04-16T15:40:29.769778+0000","last_fullsized":"2024-04-16T15:40:29.769778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918912+0000","last_change":"2024-04-16T15:40:28.571485+0000","last_active":"2024-04-16T15:40:29.918912+0000","last_peered":"2024-04-16T15:40:29.918912+0000","last_clean":"2024-04-16T15:40:29.918912+0000","last_became_active":"2024-04-16T15:40:28.570667+0000","last_became_peered":"2024-04-16T15:40:28.570667+0000","last_unstale":"2024-04-16T15:40:29.918912+0000","last_undegraded":"2024-04-16T15:40:29.918912+0000","last_fullsized":"2024-04-16T15:40:29.918912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:04.979453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920895+0000","last_change":"2024-04-16T15:40:24.548267+0000","last_active":"2024-04-16T15:40:29.920895+0000","last_peered":"2024-04-16T15:40:29.920895+0000","last_clean":"2024-04-16T15:40:29.920895+0000","last_became_active":"2024-04-16T15:40:24.547973+0000","last_became_peered":"2024-04-16T15:40:24.547973+0000","last_unstale":"2024-04-16T15:40:29.920895+0000","last_undegraded":"2024-04-16T15:40:29.920895+0000","last_fullsized":"2024-04-16T15:40:29.920895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:57:50.186857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566718+0000","last_change":"2024-04-16T15:40:23.523969+0000","last_active":"2024-04-16T15:40:29.566718+0000","last_peered":"2024-04-16T15:40:29.566718+0000","last_clean":"2024-04-16T15:40:29.566718+0000","last_became_active":"2024-04-16T15:40:23.523676+0000","last_became_peered":"2024-04-16T15:40:23.523676+0000","last_unstale":"2024-04-16T15:40:29.566718+0000","last_undegraded":"2024-04-16T15:40:29.566718+0000","last_fullsized":"2024-04-16T15:40:29.566718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919411+0000","last_change":"2024-04-16T15:40:28.807519+0000","last_active":"2024-04-16T15:40:29.919411+0000","last_peered":"2024-04-16T15:40:29.919411+0000","last_clean":"2024-04-16T15:40:29.919411+0000","last_became_active":"2024-04-16T15:40:28.807122+0000","last_became_peered":"2024-04-16T15:40:28.807122+0000","last_unstale":"2024-04-16T15:40:29.919411+0000","last_undegraded":"2024-04-16T15:40:29.919411+0000","last_fullsized":"2024-04-16T15:40:29.919411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:52:28.865900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920489+0000","last_change":"2024-04-16T15:40:24.545147+0000","last_active":"2024-04-16T15:40:29.920489+0000","last_peered":"2024-04-16T15:40:29.920489+0000","last_clean":"2024-04-16T15:40:29.920489+0000","last_became_active":"2024-04-16T15:40:24.544857+0000","last_became_peered":"2024-04-16T15:40:24.544857+0000","last_unstale":"2024-04-16T15:40:29.920489+0000","last_undegraded":"2024-04-16T15:40:29.920489+0000","last_fullsized":"2024-04-16T15:40:29.920489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:31.030716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920619+0000","last_change":"2024-04-16T15:40:23.536928+0000","last_active":"2024-04-16T15:40:29.920619+0000","last_peered":"2024-04-16T15:40:29.920619+0000","last_clean":"2024-04-16T15:40:29.920619+0000","last_became_active":"2024-04-16T15:40:23.535290+0000","last_became_peered":"2024-04-16T15:40:23.535290+0000","last_unstale":"2024-04-16T15:40:29.920619+0000","last_undegraded":"2024-04-16T15:40:29.920619+0000","last_fullsized":"2024-04-16T15:40:29.920619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:26.916980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919286+0000","last_change":"2024-04-16T15:40:28.561919+0000","last_active":"2024-04-16T15:40:29.919286+0000","last_peered":"2024-04-16T15:40:29.919286+0000","last_clean":"2024-04-16T15:40:29.919286+0000","last_became_active":"2024-04-16T15:40:28.561491+0000","last_became_peered":"2024-04-16T15:40:28.561491+0000","last_unstale":"2024-04-16T15:40:29.919286+0000","last_undegraded":"2024-04-16T15:40:29.919286+0000","last_fullsized":"2024-04-16T15:40:29.919286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:19.925886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919818+0000","last_change":"2024-04-16T15:40:24.544255+0000","last_active":"2024-04-16T15:40:29.919818+0000","last_peered":"2024-04-16T15:40:29.919818+0000","last_clean":"2024-04-16T15:40:29.919818+0000","last_became_active":"2024-04-16T15:40:24.543995+0000","last_became_peered":"2024-04-16T15:40:24.543995+0000","last_unstale":"2024-04-16T15:40:29.919818+0000","last_undegraded":"2024-04-16T15:40:29.919818+0000","last_fullsized":"2024-04-16T15:40:29.919818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:28.169445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566390+0000","last_change":"2024-04-16T15:40:23.530042+0000","last_active":"2024-04-16T15:40:29.566390+0000","last_peered":"2024-04-16T15:40:29.566390+0000","last_clean":"2024-04-16T15:40:29.566390+0000","last_became_active":"2024-04-16T15:40:23.528762+0000","last_became_peered":"2024-04-16T15:40:23.528762+0000","last_unstale":"2024-04-16T15:40:29.566390+0000","last_undegraded":"2024-04-16T15:40:29.566390+0000","last_fullsized":"2024-04-16T15:40:29.566390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919066+0000","last_change":"2024-04-16T15:40:28.568830+0000","last_active":"2024-04-16T15:40:29.919066+0000","last_peered":"2024-04-16T15:40:29.919066+0000","last_clean":"2024-04-16T15:40:29.919066+0000","last_became_active":"2024-04-16T15:40:28.568395+0000","last_became_peered":"2024-04-16T15:40:28.568395+0000","last_unstale":"2024-04-16T15:40:29.919066+0000","last_undegraded":"2024-04-16T15:40:29.919066+0000","last_fullsized":"2024-04-16T15:40:29.919066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:42.690952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566645+0000","last_change":"2024-04-16T15:40:24.543975+0000","last_active":"2024-04-16T15:40:29.566645+0000","last_peered":"2024-04-16T15:40:29.566645+0000","last_clean":"2024-04-16T15:40:29.566645+0000","last_became_active":"2024-04-16T15:40:24.543720+0000","last_became_peered":"2024-04-16T15:40:24.543720+0000","last_unstale":"2024-04-16T15:40:29.566645+0000","last_undegraded":"2024-04-16T15:40:29.566645+0000","last_fullsized":"2024-04-16T15:40:29.566645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566771+0000","last_change":"2024-04-16T15:40:23.536758+0000","last_active":"2024-04-16T15:40:29.566771+0000","last_peered":"2024-04-16T15:40:29.566771+0000","last_clean":"2024-04-16T15:40:29.566771+0000","last_became_active":"2024-04-16T15:40:23.536479+0000","last_became_peered":"2024-04-16T15:40:23.536479+0000","last_unstale":"2024-04-16T15:40:29.566771+0000","last_undegraded":"2024-04-16T15:40:29.566771+0000","last_fullsized":"2024-04-16T15:40:29.566771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567798+0000","last_change":"2024-04-16T15:40:28.807067+0000","last_active":"2024-04-16T15:40:29.567798+0000","last_peered":"2024-04-16T15:40:29.567798+0000","last_clean":"2024-04-16T15:40:29.567798+0000","last_became_active":"2024-04-16T15:40:28.805886+0000","last_became_peered":"2024-04-16T15:40:28.805886+0000","last_unstale":"2024-04-16T15:40:29.567798+0000","last_undegraded":"2024-04-16T15:40:29.567798+0000","last_fullsized":"2024-04-16T15:40:29.567798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:52.503196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920077+0000","last_change":"2024-04-16T15:40:24.546841+0000","last_active":"2024-04-16T15:40:29.920077+0000","last_peered":"2024-04-16T15:40:29.920077+0000","last_clean":"2024-04-16T15:40:29.920077+0000","last_became_active":"2024-04-16T15:40:24.546578+0000","last_became_peered":"2024-04-16T15:40:24.546578+0000","last_unstale":"2024-04-16T15:40:29.920077+0000","last_undegraded":"2024-04-16T15:40:29.920077+0000","last_fullsized":"2024-04-16T15:40:29.920077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:47:00.952582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920432+0000","last_change":"2024-04-16T15:40:23.527740+0000","last_active":"2024-04-16T15:40:29.920432+0000","last_peered":"2024-04-16T15:40:29.920432+0000","last_clean":"2024-04-16T15:40:29.920432+0000","last_became_active":"2024-04-16T15:40:23.527465+0000","last_became_peered":"2024-04-16T15:40:23.527465+0000","last_unstale":"2024-04-16T15:40:29.920432+0000","last_undegraded":"2024-04-16T15:40:29.920432+0000","last_fullsized":"2024-04-16T15:40:29.920432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:53:49.137705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567222+0000","last_change":"2024-04-16T15:40:28.804586+0000","last_active":"2024-04-16T15:40:29.567222+0000","last_peered":"2024-04-16T15:40:29.567222+0000","last_clean":"2024-04-16T15:40:29.567222+0000","last_became_active":"2024-04-16T15:40:28.804188+0000","last_became_peered":"2024-04-16T15:40:28.804188+0000","last_unstale":"2024-04-16T15:40:29.567222+0000","last_undegraded":"2024-04-16T15:40:29.567222+0000","last_fullsized":"2024-04-16T15:40:29.567222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:54:44.467991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918678+0000","last_change":"2024-04-16T15:40:24.548554+0000","last_active":"2024-04-16T15:40:29.918678+0000","last_peered":"2024-04-16T15:40:29.918678+0000","last_clean":"2024-04-16T15:40:29.918678+0000","last_became_active":"2024-04-16T15:40:24.548292+0000","last_became_peered":"2024-04-16T15:40:24.548292+0000","last_unstale":"2024-04-16T15:40:29.918678+0000","last_undegraded":"2024-04-16T15:40:29.918678+0000","last_fullsized":"2024-04-16T15:40:29.918678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:16:10.906173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918961+0000","last_change":"2024-04-16T15:40:23.537872+0000","last_active":"2024-04-16T15:40:29.918961+0000","last_peered":"2024-04-16T15:40:29.918961+0000","last_clean":"2024-04-16T15:40:29.918961+0000","last_became_active":"2024-04-16T15:40:23.537586+0000","last_became_peered":"2024-04-16T15:40:23.537586+0000","last_unstale":"2024-04-16T15:40:29.918961+0000","last_undegraded":"2024-04-16T15:40:29.918961+0000","last_fullsized":"2024-04-16T15:40:29.918961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:10:35.227712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920389+0000","last_change":"2024-04-16T15:40:28.805253+0000","last_active":"2024-04-16T15:40:29.920389+0000","last_peered":"2024-04-16T15:40:29.920389+0000","last_clean":"2024-04-16T15:40:29.920389+0000","last_became_active":"2024-04-16T15:40:28.804927+0000","last_became_peered":"2024-04-16T15:40:28.804927+0000","last_unstale":"2024-04-16T15:40:29.920389+0000","last_undegraded":"2024-04-16T15:40:29.920389+0000","last_fullsized":"2024-04-16T15:40:29.920389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:55.036666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921526+0000","last_change":"2024-04-16T15:40:24.549880+0000","last_active":"2024-04-16T15:40:29.921526+0000","last_peered":"2024-04-16T15:40:29.921526+0000","last_clean":"2024-04-16T15:40:29.921526+0000","last_became_active":"2024-04-16T15:40:24.549176+0000","last_became_peered":"2024-04-16T15:40:24.549176+0000","last_unstale":"2024-04-16T15:40:29.921526+0000","last_undegraded":"2024-04-16T15:40:29.921526+0000","last_fullsized":"2024-04-16T15:40:29.921526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:32.578262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:34.090660+0000","last_change":"2024-04-16T15:40:23.528022+0000","last_active":"2024-04-16T15:40:34.090660+0000","last_peered":"2024-04-16T15:40:34.090660+0000","last_clean":"2024-04-16T15:40:34.090660+0000","last_became_active":"2024-04-16T15:40:23.527723+0000","last_became_peered":"2024-04-16T15:40:23.527723+0000","last_unstale":"2024-04-16T15:40:34.090660+0000","last_undegraded":"2024-04-16T15:40:34.090660+0000","last_fullsized":"2024-04-16T15:40:34.090660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769345+0000","last_change":"2024-04-16T15:40:28.570589+0000","last_active":"2024-04-16T15:40:29.769345+0000","last_peered":"2024-04-16T15:40:29.769345+0000","last_clean":"2024-04-16T15:40:29.769345+0000","last_became_active":"2024-04-16T15:40:28.570223+0000","last_became_peered":"2024-04-16T15:40:28.570223+0000","last_unstale":"2024-04-16T15:40:29.769345+0000","last_undegraded":"2024-04-16T15:40:29.769345+0000","last_fullsized":"2024-04-16T15:40:29.769345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:35.057588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920264+0000","last_change":"2024-04-16T15:40:24.548751+0000","last_active":"2024-04-16T15:40:29.920264+0000","last_peered":"2024-04-16T15:40:29.920264+0000","last_clean":"2024-04-16T15:40:29.920264+0000","last_became_active":"2024-04-16T15:40:24.548459+0000","last_became_peered":"2024-04-16T15:40:24.548459+0000","last_unstale":"2024-04-16T15:40:29.920264+0000","last_undegraded":"2024-04-16T15:40:29.920264+0000","last_fullsized":"2024-04-16T15:40:29.920264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:50:22.947852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769687+0000","last_change":"2024-04-16T15:40:23.531085+0000","last_active":"2024-04-16T15:40:29.769687+0000","last_peered":"2024-04-16T15:40:29.769687+0000","last_clean":"2024-04-16T15:40:29.769687+0000","last_became_active":"2024-04-16T15:40:23.530814+0000","last_became_peered":"2024-04-16T15:40:23.530814+0000","last_unstale":"2024-04-16T15:40:29.769687+0000","last_undegraded":"2024-04-16T15:40:29.769687+0000","last_fullsized":"2024-04-16T15:40:29.769687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920980+0000","last_change":"2024-04-16T15:40:28.570865+0000","last_active":"2024-04-16T15:40:29.920980+0000","last_peered":"2024-04-16T15:40:29.920980+0000","last_clean":"2024-04-16T15:40:29.920980+0000","last_became_active":"2024-04-16T15:40:28.570475+0000","last_became_peered":"2024-04-16T15:40:28.570475+0000","last_unstale":"2024-04-16T15:40:29.920980+0000","last_undegraded":"2024-04-16T15:40:29.920980+0000","last_fullsized":"2024-04-16T15:40:29.920980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:32.196316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919181+0000","last_change":"2024-04-16T15:40:24.543283+0000","last_active":"2024-04-16T15:40:29.919181+0000","last_peered":"2024-04-16T15:40:29.919181+0000","last_clean":"2024-04-16T15:40:29.919181+0000","last_became_active":"2024-04-16T15:40:24.542708+0000","last_became_peered":"2024-04-16T15:40:24.542708+0000","last_unstale":"2024-04-16T15:40:29.919181+0000","last_undegraded":"2024-04-16T15:40:29.919181+0000","last_fullsized":"2024-04-16T15:40:29.919181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920195+0000","last_change":"2024-04-16T15:40:23.530402+0000","last_active":"2024-04-16T15:40:29.920195+0000","last_peered":"2024-04-16T15:40:29.920195+0000","last_clean":"2024-04-16T15:40:29.920195+0000","last_became_active":"2024-04-16T15:40:23.530122+0000","last_became_peered":"2024-04-16T15:40:23.530122+0000","last_unstale":"2024-04-16T15:40:29.920195+0000","last_undegraded":"2024-04-16T15:40:29.920195+0000","last_fullsized":"2024-04-16T15:40:29.920195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:10:25.545389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566298+0000","last_change":"2024-04-16T15:40:28.563586+0000","last_active":"2024-04-16T15:40:29.566298+0000","last_peered":"2024-04-16T15:40:29.566298+0000","last_clean":"2024-04-16T15:40:29.566298+0000","last_became_active":"2024-04-16T15:40:28.563187+0000","last_became_peered":"2024-04-16T15:40:28.563187+0000","last_unstale":"2024-04-16T15:40:29.566298+0000","last_undegraded":"2024-04-16T15:40:29.566298+0000","last_fullsized":"2024-04-16T15:40:29.566298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:28.407124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919923+0000","last_change":"2024-04-16T15:40:24.550624+0000","last_active":"2024-04-16T15:40:29.919923+0000","last_peered":"2024-04-16T15:40:29.919923+0000","last_clean":"2024-04-16T15:40:29.919923+0000","last_became_active":"2024-04-16T15:40:24.550410+0000","last_became_peered":"2024-04-16T15:40:24.550410+0000","last_unstale":"2024-04-16T15:40:29.919923+0000","last_undegraded":"2024-04-16T15:40:29.919923+0000","last_fullsized":"2024-04-16T15:40:29.919923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:32.578262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567285+0000","last_change":"2024-04-16T15:40:23.531043+0000","last_active":"2024-04-16T15:40:29.567285+0000","last_peered":"2024-04-16T15:40:29.567285+0000","last_clean":"2024-04-16T15:40:29.567285+0000","last_became_active":"2024-04-16T15:40:23.530764+0000","last_became_peered":"2024-04-16T15:40:23.530764+0000","last_unstale":"2024-04-16T15:40:29.567285+0000","last_undegraded":"2024-04-16T15:40:29.567285+0000","last_fullsized":"2024-04-16T15:40:29.567285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:44.868086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566507+0000","last_change":"2024-04-16T15:40:28.805696+0000","last_active":"2024-04-16T15:40:29.566507+0000","last_peered":"2024-04-16T15:40:29.566507+0000","last_clean":"2024-04-16T15:40:29.566507+0000","last_became_active":"2024-04-16T15:40:28.805319+0000","last_became_peered":"2024-04-16T15:40:28.805319+0000","last_unstale":"2024-04-16T15:40:29.566507+0000","last_undegraded":"2024-04-16T15:40:29.566507+0000","last_fullsized":"2024-04-16T15:40:29.566507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:55.036666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918914+0000","last_change":"2024-04-16T15:40:28.804646+0000","last_active":"2024-04-16T15:40:29.918914+0000","last_peered":"2024-04-16T15:40:29.918914+0000","last_clean":"2024-04-16T15:40:29.918914+0000","last_became_active":"2024-04-16T15:40:28.804239+0000","last_became_peered":"2024-04-16T15:40:28.804239+0000","last_unstale":"2024-04-16T15:40:29.918914+0000","last_undegraded":"2024-04-16T15:40:29.918914+0000","last_fullsized":"2024-04-16T15:40:29.918914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:15:36.605133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769103+0000","last_change":"2024-04-16T15:40:24.533204+0000","last_active":"2024-04-16T15:40:29.769103+0000","last_peered":"2024-04-16T15:40:29.769103+0000","last_clean":"2024-04-16T15:40:29.769103+0000","last_became_active":"2024-04-16T15:40:24.532947+0000","last_became_peered":"2024-04-16T15:40:24.532947+0000","last_unstale":"2024-04-16T15:40:29.769103+0000","last_undegraded":"2024-04-16T15:40:29.769103+0000","last_fullsized":"2024-04-16T15:40:29.769103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919749+0000","last_change":"2024-04-16T15:40:23.537693+0000","last_active":"2024-04-16T15:40:29.919749+0000","last_peered":"2024-04-16T15:40:29.919749+0000","last_clean":"2024-04-16T15:40:29.919749+0000","last_became_active":"2024-04-16T15:40:23.536396+0000","last_became_peered":"2024-04-16T15:40:23.536396+0000","last_unstale":"2024-04-16T15:40:29.919749+0000","last_undegraded":"2024-04-16T15:40:29.919749+0000","last_fullsized":"2024-04-16T15:40:29.919749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:23:15.857720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567324+0000","last_change":"2024-04-16T15:40:28.567889+0000","last_active":"2024-04-16T15:40:29.567324+0000","last_peered":"2024-04-16T15:40:29.567324+0000","last_clean":"2024-04-16T15:40:29.567324+0000","last_became_active":"2024-04-16T15:40:28.567519+0000","last_became_peered":"2024-04-16T15:40:28.567519+0000","last_unstale":"2024-04-16T15:40:29.567324+0000","last_undegraded":"2024-04-16T15:40:29.567324+0000","last_fullsized":"2024-04-16T15:40:29.567324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:29.971423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566216+0000","last_change":"2024-04-16T15:40:24.550785+0000","last_active":"2024-04-16T15:40:29.566216+0000","last_peered":"2024-04-16T15:40:29.566216+0000","last_clean":"2024-04-16T15:40:29.566216+0000","last_became_active":"2024-04-16T15:40:24.550492+0000","last_became_peered":"2024-04-16T15:40:24.550492+0000","last_unstale":"2024-04-16T15:40:29.566216+0000","last_undegraded":"2024-04-16T15:40:29.566216+0000","last_fullsized":"2024-04-16T15:40:29.566216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567393+0000","last_change":"2024-04-16T15:40:23.527649+0000","last_active":"2024-04-16T15:40:29.567393+0000","last_peered":"2024-04-16T15:40:29.567393+0000","last_clean":"2024-04-16T15:40:29.567393+0000","last_became_active":"2024-04-16T15:40:23.526805+0000","last_became_peered":"2024-04-16T15:40:23.526805+0000","last_unstale":"2024-04-16T15:40:29.567393+0000","last_undegraded":"2024-04-16T15:40:29.567393+0000","last_fullsized":"2024-04-16T15:40:29.567393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:17.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566914+0000","last_change":"2024-04-16T15:40:23.536732+0000","last_active":"2024-04-16T15:40:29.566914+0000","last_peered":"2024-04-16T15:40:29.566914+0000","last_clean":"2024-04-16T15:40:29.566914+0000","last_became_active":"2024-04-16T15:40:23.536431+0000","last_became_peered":"2024-04-16T15:40:23.536431+0000","last_unstale":"2024-04-16T15:40:29.566914+0000","last_undegraded":"2024-04-16T15:40:29.566914+0000","last_fullsized":"2024-04-16T15:40:29.566914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918955+0000","last_change":"2024-04-16T15:40:24.538554+0000","last_active":"2024-04-16T15:40:29.918955+0000","last_peered":"2024-04-16T15:40:29.918955+0000","last_clean":"2024-04-16T15:40:29.918955+0000","last_became_active":"2024-04-16T15:40:24.538271+0000","last_became_peered":"2024-04-16T15:40:24.538271+0000","last_unstale":"2024-04-16T15:40:29.918955+0000","last_undegraded":"2024-04-16T15:40:29.918955+0000","last_fullsized":"2024-04-16T15:40:29.918955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567207+0000","last_change":"2024-04-16T15:40:28.566595+0000","last_active":"2024-04-16T15:40:29.567207+0000","last_peered":"2024-04-16T15:40:29.567207+0000","last_clean":"2024-04-16T15:40:29.567207+0000","last_became_active":"2024-04-16T15:40:28.566177+0000","last_became_peered":"2024-04-16T15:40:28.566177+0000","last_unstale":"2024-04-16T15:40:29.567207+0000","last_undegraded":"2024-04-16T15:40:29.567207+0000","last_fullsized":"2024-04-16T15:40:29.567207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:34.092734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567347+0000","last_change":"2024-04-16T15:40:23.528855+0000","last_active":"2024-04-16T15:40:29.567347+0000","last_peered":"2024-04-16T15:40:29.567347+0000","last_clean":"2024-04-16T15:40:29.567347+0000","last_became_active":"2024-04-16T15:40:23.528616+0000","last_became_peered":"2024-04-16T15:40:23.528616+0000","last_unstale":"2024-04-16T15:40:29.567347+0000","last_undegraded":"2024-04-16T15:40:29.567347+0000","last_fullsized":"2024-04-16T15:40:29.567347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:22.508184+0000","last_clean_scrub_stamp":"2024-04-16T15:40:22.508184+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:05.645946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919604+0000","last_change":"2024-04-16T15:40:24.535332+0000","last_active":"2024-04-16T15:40:29.919604+0000","last_peered":"2024-04-16T15:40:29.919604+0000","last_clean":"2024-04-16T15:40:29.919604+0000","last_became_active":"2024-04-16T15:40:24.535031+0000","last_became_peered":"2024-04-16T15:40:24.535031+0000","last_unstale":"2024-04-16T15:40:29.919604+0000","last_undegraded":"2024-04-16T15:40:29.919604+0000","last_fullsized":"2024-04-16T15:40:29.919604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:46:45.872510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567748+0000","last_change":"2024-04-16T15:40:28.564309+0000","last_active":"2024-04-16T15:40:29.567748+0000","last_peered":"2024-04-16T15:40:29.567748+0000","last_clean":"2024-04-16T15:40:29.567748+0000","last_became_active":"2024-04-16T15:40:28.563916+0000","last_became_peered":"2024-04-16T15:40:28.563916+0000","last_unstale":"2024-04-16T15:40:29.567748+0000","last_undegraded":"2024-04-16T15:40:29.567748+0000","last_fullsized":"2024-04-16T15:40:29.567748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:27.539480+0000","last_clean_scrub_stamp":"2024-04-16T15:40:27.539480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:02:28.301124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566423+0000","last_change":"2024-04-16T15:40:24.543242+0000","last_active":"2024-04-16T15:40:29.566423+0000","last_peered":"2024-04-16T15:40:29.566423+0000","last_clean":"2024-04-16T15:40:29.566423+0000","last_became_active":"2024-04-16T15:40:24.542947+0000","last_became_peered":"2024-04-16T15:40:24.542947+0000","last_unstale":"2024-04-16T15:40:29.566423+0000","last_undegraded":"2024-04-16T15:40:29.566423+0000","last_fullsized":"2024-04-16T15:40:29.566423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566822+0000","last_change":"2024-04-16T15:40:24.535196+0000","last_active":"2024-04-16T15:40:29.566822+0000","last_peered":"2024-04-16T15:40:29.566822+0000","last_clean":"2024-04-16T15:40:29.566822+0000","last_became_active":"2024-04-16T15:40:24.534885+0000","last_became_peered":"2024-04-16T15:40:24.534885+0000","last_unstale":"2024-04-16T15:40:29.566822+0000","last_undegraded":"2024-04-16T15:40:29.566822+0000","last_fullsized":"2024-04-16T15:40:29.566822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:20.979138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920587+0000","last_change":"2024-04-16T15:40:24.548666+0000","last_active":"2024-04-16T15:40:29.920587+0000","last_peered":"2024-04-16T15:40:29.920587+0000","last_clean":"2024-04-16T15:40:29.920587+0000","last_became_active":"2024-04-16T15:40:24.548388+0000","last_became_peered":"2024-04-16T15:40:24.548388+0000","last_unstale":"2024-04-16T15:40:29.920587+0000","last_undegraded":"2024-04-16T15:40:29.920587+0000","last_fullsized":"2024-04-16T15:40:29.920587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:56:28.169445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918853+0000","last_change":"2024-04-16T15:40:24.542842+0000","last_active":"2024-04-16T15:40:29.918853+0000","last_peered":"2024-04-16T15:40:29.918853+0000","last_clean":"2024-04-16T15:40:29.918853+0000","last_became_active":"2024-04-16T15:40:24.542514+0000","last_became_peered":"2024-04-16T15:40:24.542514+0000","last_unstale":"2024-04-16T15:40:29.918853+0000","last_undegraded":"2024-04-16T15:40:29.918853+0000","last_fullsized":"2024-04-16T15:40:29.918853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567390+0000","last_change":"2024-04-16T15:40:24.543863+0000","last_active":"2024-04-16T15:40:29.567390+0000","last_peered":"2024-04-16T15:40:29.567390+0000","last_clean":"2024-04-16T15:40:29.567390+0000","last_became_active":"2024-04-16T15:40:24.543611+0000","last_became_peered":"2024-04-16T15:40:24.543611+0000","last_unstale":"2024-04-16T15:40:29.567390+0000","last_undegraded":"2024-04-16T15:40:29.567390+0000","last_fullsized":"2024-04-16T15:40:29.567390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769571+0000","last_change":"2024-04-16T15:40:24.533049+0000","last_active":"2024-04-16T15:40:29.769571+0000","last_peered":"2024-04-16T15:40:29.769571+0000","last_clean":"2024-04-16T15:40:29.769571+0000","last_became_active":"2024-04-16T15:40:24.532782+0000","last_became_peered":"2024-04-16T15:40:24.532782+0000","last_unstale":"2024-04-16T15:40:29.769571+0000","last_undegraded":"2024-04-16T15:40:29.769571+0000","last_fullsized":"2024-04-16T15:40:29.769571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567235+0000","last_change":"2024-04-16T15:40:24.546110+0000","last_active":"2024-04-16T15:40:29.567235+0000","last_peered":"2024-04-16T15:40:29.567235+0000","last_clean":"2024-04-16T15:40:29.567235+0000","last_became_active":"2024-04-16T15:40:24.545660+0000","last_became_peered":"2024-04-16T15:40:24.545660+0000","last_unstale":"2024-04-16T15:40:29.567235+0000","last_undegraded":"2024-04-16T15:40:29.567235+0000","last_fullsized":"2024-04-16T15:40:29.567235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567701+0000","last_change":"2024-04-16T15:40:24.543839+0000","last_active":"2024-04-16T15:40:29.567701+0000","last_peered":"2024-04-16T15:40:29.567701+0000","last_clean":"2024-04-16T15:40:29.567701+0000","last_became_active":"2024-04-16T15:40:24.543560+0000","last_became_peered":"2024-04-16T15:40:24.543560+0000","last_unstale":"2024-04-16T15:40:29.567701+0000","last_undegraded":"2024-04-16T15:40:29.567701+0000","last_fullsized":"2024-04-16T15:40:29.567701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919650+0000","last_change":"2024-04-16T15:40:24.533578+0000","last_active":"2024-04-16T15:40:29.919650+0000","last_peered":"2024-04-16T15:40:29.919650+0000","last_clean":"2024-04-16T15:40:29.919650+0000","last_became_active":"2024-04-16T15:40:24.533335+0000","last_became_peered":"2024-04-16T15:40:24.533335+0000","last_unstale":"2024-04-16T15:40:29.919650+0000","last_undegraded":"2024-04-16T15:40:29.919650+0000","last_fullsized":"2024-04-16T15:40:29.919650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567192+0000","last_change":"2024-04-16T15:40:24.545340+0000","last_active":"2024-04-16T15:40:29.567192+0000","last_peered":"2024-04-16T15:40:29.567192+0000","last_clean":"2024-04-16T15:40:29.567192+0000","last_became_active":"2024-04-16T15:40:24.545096+0000","last_became_peered":"2024-04-16T15:40:24.545096+0000","last_unstale":"2024-04-16T15:40:29.567192+0000","last_undegraded":"2024-04-16T15:40:29.567192+0000","last_fullsized":"2024-04-16T15:40:29.567192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920042+0000","last_change":"2024-04-16T15:40:24.551716+0000","last_active":"2024-04-16T15:40:29.920042+0000","last_peered":"2024-04-16T15:40:29.920042+0000","last_clean":"2024-04-16T15:40:29.920042+0000","last_became_active":"2024-04-16T15:40:24.551360+0000","last_became_peered":"2024-04-16T15:40:24.551360+0000","last_unstale":"2024-04-16T15:40:29.920042+0000","last_undegraded":"2024-04-16T15:40:29.920042+0000","last_fullsized":"2024-04-16T15:40:29.920042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:14.076265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918705+0000","last_change":"2024-04-16T15:40:24.549554+0000","last_active":"2024-04-16T15:40:29.918705+0000","last_peered":"2024-04-16T15:40:29.918705+0000","last_clean":"2024-04-16T15:40:29.918705+0000","last_became_active":"2024-04-16T15:40:24.548848+0000","last_became_peered":"2024-04-16T15:40:24.548848+0000","last_unstale":"2024-04-16T15:40:29.918705+0000","last_undegraded":"2024-04-16T15:40:29.918705+0000","last_fullsized":"2024-04-16T15:40:29.918705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567025+0000","last_change":"2024-04-16T15:40:24.539612+0000","last_active":"2024-04-16T15:40:29.567025+0000","last_peered":"2024-04-16T15:40:29.567025+0000","last_clean":"2024-04-16T15:40:29.567025+0000","last_became_active":"2024-04-16T15:40:24.536911+0000","last_became_peered":"2024-04-16T15:40:24.536911+0000","last_unstale":"2024-04-16T15:40:29.567025+0000","last_undegraded":"2024-04-16T15:40:29.567025+0000","last_fullsized":"2024-04-16T15:40:29.567025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:37:17.972187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921716+0000","last_change":"2024-04-16T15:40:24.550620+0000","last_active":"2024-04-16T15:40:29.921716+0000","last_peered":"2024-04-16T15:40:29.921716+0000","last_clean":"2024-04-16T15:40:29.921716+0000","last_became_active":"2024-04-16T15:40:24.549587+0000","last_became_peered":"2024-04-16T15:40:24.549587+0000","last_unstale":"2024-04-16T15:40:29.921716+0000","last_undegraded":"2024-04-16T15:40:29.921716+0000","last_fullsized":"2024-04-16T15:40:29.921716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:27.826954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566610+0000","last_change":"2024-04-16T15:40:24.539758+0000","last_active":"2024-04-16T15:40:29.566610+0000","last_peered":"2024-04-16T15:40:29.566610+0000","last_clean":"2024-04-16T15:40:29.566610+0000","last_became_active":"2024-04-16T15:40:24.539525+0000","last_became_peered":"2024-04-16T15:40:24.539525+0000","last_unstale":"2024-04-16T15:40:29.566610+0000","last_undegraded":"2024-04-16T15:40:29.566610+0000","last_fullsized":"2024-04-16T15:40:29.566610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:25:44.286135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.921033+0000","last_change":"2024-04-16T15:40:24.548867+0000","last_active":"2024-04-16T15:40:29.921033+0000","last_peered":"2024-04-16T15:40:29.921033+0000","last_clean":"2024-04-16T15:40:29.921033+0000","last_became_active":"2024-04-16T15:40:24.548585+0000","last_became_peered":"2024-04-16T15:40:24.548585+0000","last_unstale":"2024-04-16T15:40:29.921033+0000","last_undegraded":"2024-04-16T15:40:29.921033+0000","last_fullsized":"2024-04-16T15:40:29.921033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:47:31.030716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919546+0000","last_change":"2024-04-16T15:40:24.545697+0000","last_active":"2024-04-16T15:40:29.919546+0000","last_peered":"2024-04-16T15:40:29.919546+0000","last_clean":"2024-04-16T15:40:29.919546+0000","last_became_active":"2024-04-16T15:40:24.545409+0000","last_became_peered":"2024-04-16T15:40:24.545409+0000","last_unstale":"2024-04-16T15:40:29.919546+0000","last_undegraded":"2024-04-16T15:40:29.919546+0000","last_fullsized":"2024-04-16T15:40:29.919546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:17:35.981376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769173+0000","last_change":"2024-04-16T15:40:24.544806+0000","last_active":"2024-04-16T15:40:29.769173+0000","last_peered":"2024-04-16T15:40:29.769173+0000","last_clean":"2024-04-16T15:40:29.769173+0000","last_became_active":"2024-04-16T15:40:24.544550+0000","last_became_peered":"2024-04-16T15:40:24.544550+0000","last_unstale":"2024-04-16T15:40:29.769173+0000","last_undegraded":"2024-04-16T15:40:29.769173+0000","last_fullsized":"2024-04-16T15:40:29.769173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918672+0000","last_change":"2024-04-16T15:40:24.542813+0000","last_active":"2024-04-16T15:40:29.918672+0000","last_peered":"2024-04-16T15:40:29.918672+0000","last_clean":"2024-04-16T15:40:29.918672+0000","last_became_active":"2024-04-16T15:40:24.534241+0000","last_became_peered":"2024-04-16T15:40:24.534241+0000","last_unstale":"2024-04-16T15:40:29.918672+0000","last_undegraded":"2024-04-16T15:40:29.918672+0000","last_fullsized":"2024-04-16T15:40:29.918672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:02:30.065862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567152+0000","last_change":"2024-04-16T15:40:24.546408+0000","last_active":"2024-04-16T15:40:29.567152+0000","last_peered":"2024-04-16T15:40:29.567152+0000","last_clean":"2024-04-16T15:40:29.567152+0000","last_became_active":"2024-04-16T15:40:24.546168+0000","last_became_peered":"2024-04-16T15:40:24.546168+0000","last_unstale":"2024-04-16T15:40:29.567152+0000","last_undegraded":"2024-04-16T15:40:29.567152+0000","last_fullsized":"2024-04-16T15:40:29.567152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:36:24.222201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920159+0000","last_change":"2024-04-16T15:40:24.533057+0000","last_active":"2024-04-16T15:40:29.920159+0000","last_peered":"2024-04-16T15:40:29.920159+0000","last_clean":"2024-04-16T15:40:29.920159+0000","last_became_active":"2024-04-16T15:40:24.532732+0000","last_became_peered":"2024-04-16T15:40:24.532732+0000","last_unstale":"2024-04-16T15:40:29.920159+0000","last_undegraded":"2024-04-16T15:40:29.920159+0000","last_fullsized":"2024-04-16T15:40:29.920159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:13.766871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567143+0000","last_change":"2024-04-16T15:40:24.540869+0000","last_active":"2024-04-16T15:40:29.567143+0000","last_peered":"2024-04-16T15:40:29.567143+0000","last_clean":"2024-04-16T15:40:29.567143+0000","last_became_active":"2024-04-16T15:40:24.537498+0000","last_became_peered":"2024-04-16T15:40:24.537498+0000","last_unstale":"2024-04-16T15:40:29.567143+0000","last_undegraded":"2024-04-16T15:40:29.567143+0000","last_fullsized":"2024-04-16T15:40:29.567143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.567216+0000","last_change":"2024-04-16T15:40:24.536207+0000","last_active":"2024-04-16T15:40:29.567216+0000","last_peered":"2024-04-16T15:40:29.567216+0000","last_clean":"2024-04-16T15:40:29.567216+0000","last_became_active":"2024-04-16T15:40:24.535940+0000","last_became_peered":"2024-04-16T15:40:24.535940+0000","last_unstale":"2024-04-16T15:40:29.567216+0000","last_undegraded":"2024-04-16T15:40:29.567216+0000","last_fullsized":"2024-04-16T15:40:29.567216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:45:15.899014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920836+0000","last_change":"2024-04-16T15:40:24.544527+0000","last_active":"2024-04-16T15:40:29.920836+0000","last_peered":"2024-04-16T15:40:29.920836+0000","last_clean":"2024-04-16T15:40:29.920836+0000","last_became_active":"2024-04-16T15:40:24.543702+0000","last_became_peered":"2024-04-16T15:40:24.543702+0000","last_unstale":"2024-04-16T15:40:29.920836+0000","last_undegraded":"2024-04-16T15:40:29.920836+0000","last_fullsized":"2024-04-16T15:40:29.920836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:24.380253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.566076+0000","last_change":"2024-04-16T15:40:24.545972+0000","last_active":"2024-04-16T15:40:29.566076+0000","last_peered":"2024-04-16T15:40:29.566076+0000","last_clean":"2024-04-16T15:40:29.566076+0000","last_became_active":"2024-04-16T15:40:24.545726+0000","last_became_peered":"2024-04-16T15:40:24.545726+0000","last_unstale":"2024-04-16T15:40:29.566076+0000","last_undegraded":"2024-04-16T15:40:29.566076+0000","last_fullsized":"2024-04-16T15:40:29.566076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:51:01.681894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919710+0000","last_change":"2024-04-16T15:40:24.550333+0000","last_active":"2024-04-16T15:40:29.919710+0000","last_peered":"2024-04-16T15:40:29.919710+0000","last_clean":"2024-04-16T15:40:29.919710+0000","last_became_active":"2024-04-16T15:40:24.550063+0000","last_became_peered":"2024-04-16T15:40:24.550063+0000","last_unstale":"2024-04-16T15:40:29.919710+0000","last_undegraded":"2024-04-16T15:40:29.919710+0000","last_fullsized":"2024-04-16T15:40:29.919710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920040+0000","last_change":"2024-04-16T15:40:24.545717+0000","last_active":"2024-04-16T15:40:29.920040+0000","last_peered":"2024-04-16T15:40:29.920040+0000","last_clean":"2024-04-16T15:40:29.920040+0000","last_became_active":"2024-04-16T15:40:24.545445+0000","last_became_peered":"2024-04-16T15:40:24.545445+0000","last_unstale":"2024-04-16T15:40:29.920040+0000","last_undegraded":"2024-04-16T15:40:29.920040+0000","last_fullsized":"2024-04-16T15:40:29.920040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:14:51.739113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919205+0000","last_change":"2024-04-16T15:40:24.549202+0000","last_active":"2024-04-16T15:40:29.919205+0000","last_peered":"2024-04-16T15:40:29.919205+0000","last_clean":"2024-04-16T15:40:29.919205+0000","last_became_active":"2024-04-16T15:40:24.548794+0000","last_became_peered":"2024-04-16T15:40:24.548794+0000","last_unstale":"2024-04-16T15:40:29.919205+0000","last_undegraded":"2024-04-16T15:40:29.919205+0000","last_fullsized":"2024-04-16T15:40:29.919205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:07:25.944552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.919742+0000","last_change":"2024-04-16T15:40:24.548634+0000","last_active":"2024-04-16T15:40:29.919742+0000","last_peered":"2024-04-16T15:40:29.919742+0000","last_clean":"2024-04-16T15:40:29.919742+0000","last_became_active":"2024-04-16T15:40:24.548342+0000","last_became_peered":"2024-04-16T15:40:24.548342+0000","last_unstale":"2024-04-16T15:40:29.919742+0000","last_undegraded":"2024-04-16T15:40:29.919742+0000","last_fullsized":"2024-04-16T15:40:29.919742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:54:18.325496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.918832+0000","last_change":"2024-04-16T15:40:24.534160+0000","last_active":"2024-04-16T15:40:29.918832+0000","last_peered":"2024-04-16T15:40:29.918832+0000","last_clean":"2024-04-16T15:40:29.918832+0000","last_became_active":"2024-04-16T15:40:24.533913+0000","last_became_peered":"2024-04-16T15:40:24.533913+0000","last_unstale":"2024-04-16T15:40:29.918832+0000","last_undegraded":"2024-04-16T15:40:29.918832+0000","last_fullsized":"2024-04-16T15:40:29.918832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:00:36.511008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.769869+0000","last_change":"2024-04-16T15:40:24.546471+0000","last_active":"2024-04-16T15:40:29.769869+0000","last_peered":"2024-04-16T15:40:29.769869+0000","last_clean":"2024-04-16T15:40:29.769869+0000","last_became_active":"2024-04-16T15:40:24.545638+0000","last_became_peered":"2024-04-16T15:40:24.545638+0000","last_unstale":"2024-04-16T15:40:29.769869+0000","last_undegraded":"2024-04-16T15:40:29.769869+0000","last_fullsized":"2024-04-16T15:40:29.769869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:44:26.516559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-16T15:40:29.920784+0000","last_change":"2024-04-16T15:40:24.546172+0000","last_active":"2024-04-16T15:40:29.920784+0000","last_peered":"2024-04-16T15:40:29.920784+0000","last_clean":"2024-04-16T15:40:29.920784+0000","last_became_active":"2024-04-16T15:40:24.545582+0000","last_became_peered":"2024-04-16T15:40:24.545582+0000","last_unstale":"2024-04-16T15:40:29.920784+0000","last_undegraded":"2024-04-16T15:40:29.920784+0000","last_fullsized":"2024-04-16T15:40:29.920784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:23.512608+0000","last_clean_scrub_stamp":"2024-04-16T15:40:23.512608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:22:21.801823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:40:44.816 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-16T15:40:44.817 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-16T15:40:44.817 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-16T15:40:44.817 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-16T15:40:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.954+0000 7f39d95f4640 1 Processor -- start 2024-04-16T15:40:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.954+0000 7f39d95f4640 1 -- start start 2024-04-16T15:40:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d95f4640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f39d4153570 0x7f39d40c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d95f4640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d40c82d0 con 0x7f39d40c4350 2024-04-16T15:40:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d95f4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d4155d70 con 0x7f39d4159740 2024-04-16T15:40:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d95f4640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f39d415a2e0 con 0x7f39d4153570 2024-04-16T15:40:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d2ffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f39d4153570 0x7f39d40c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d27fc640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f39d40c4350 0x7f39d40c5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34274/0 (socket says 172.21.15.71:34274) 2024-04-16T15:40:44.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d27fc640 1 -- 172.21.15.71:0/1915379761 learned_addr learned my addr 172.21.15.71:0/1915379761 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:44.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d2ffd640 1 -- 172.21.15.71:0/1915379761 >> v1:172.21.15.122:6789/0 conn(0x7f39d4159740 legacy=0x7f39d4159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:44.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d2ffd640 1 -- 172.21.15.71:0/1915379761 >> v1:172.21.15.71:6789/0 conn(0x7f39d40c4350 legacy=0x7f39d40c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.955+0000 7f39d2ffd640 1 -- 172.21.15.71:0/1915379761 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40c8d50 con 0x7f39d4153570 2024-04-16T15:40:44.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.956+0000 7f39d2ffd640 1 --2- 172.21.15.71:0/1915379761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d4153570 0x7f39d40c52b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39c40097a0 tx=0x7f39c402b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9a6d00de6d7385 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.956+0000 7f39d1ffb640 1 -- 172.21.15.71:0/1915379761 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4037070 con 0x7f39d4153570 2024-04-16T15:40:44.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.956+0000 7f39d1ffb640 1 -- 172.21.15.71:0/1915379761 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c4030710 con 0x7f39d4153570 2024-04-16T15:40:44.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.956+0000 7f39d1ffb640 1 -- 172.21.15.71:0/1915379761 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4030ae0 con 0x7f39d4153570 2024-04-16T15:40:44.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.957+0000 7f39d95f4640 1 -- 172.21.15.71:0/1915379761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d4153570 msgr2=0x7f39d40c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.957+0000 7f39d95f4640 1 --2- 172.21.15.71:0/1915379761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d4153570 0x7f39d40c52b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39c40097a0 tx=0x7f39c402b5e0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.957+0000 7f39d95f4640 1 -- 172.21.15.71:0/1915379761 shutdown_connections 2024-04-16T15:40:44.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.957+0000 7f39d95f4640 1 --2- 172.21.15.71:0/1915379761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d4153570 0x7f39d40c52b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.957+0000 7f39d95f4640 1 -- 172.21.15.71:0/1915379761 >> 172.21.15.71:0/1915379761 conn(0x7f39d40c96e0 msgr2=0x7f39d40c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:44.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.957+0000 7f39d95f4640 1 -- 172.21.15.71:0/1915379761 shutdown_connections 2024-04-16T15:40:44.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.957+0000 7f39d95f4640 1 -- 172.21.15.71:0/1915379761 wait complete. 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.958+0000 7f39d95f4640 1 Processor -- start 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.958+0000 7f39d95f4640 1 -- start start 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d95f4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d40c4350 0x7f39d41ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d95f4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39d4153570 0x7f39d414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d95f4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d4159740 0x7f39d4148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d95f4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40c8ff0 con 0x7f39d40c4350 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d95f4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40c64b0 con 0x7f39d4159740 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d95f4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f39d40c6630 con 0x7f39d4153570 2024-04-16T15:40:44.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d40c4350 0x7f39d41ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d40c4350 0x7f39d41ec6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47092/0 (socket says 172.21.15.71:47092) 2024-04-16T15:40:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 -- 172.21.15.71:0/267559709 learned_addr learned my addr 172.21.15.71:0/267559709 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d27fc640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39d4153570 0x7f39d414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d37fe640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d4159740 0x7f39d4148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 -- 172.21.15.71:0/267559709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39d4153570 msgr2=0x7f39d414db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39d4153570 0x7f39d414db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 -- 172.21.15.71:0/267559709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d4159740 msgr2=0x7f39d4148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:44.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d4159740 0x7f39d4148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:44.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 -- 172.21.15.71:0/267559709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d414f0b0 con 0x7f39d40c4350 2024-04-16T15:40:44.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.959+0000 7f39d2ffd640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d40c4350 0x7f39d41ec6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f39c40097a0 tx=0x7f39c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.960+0000 7f39bb7fe640 1 -- 172.21.15.71:0/267559709 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4042070 con 0x7f39d40c4350 2024-04-16T15:40:44.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.960+0000 7f39bb7fe640 1 -- 172.21.15.71:0/267559709 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c4032a40 con 0x7f39d40c4350 2024-04-16T15:40:44.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.960+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d41efc00 con 0x7f39d40c4350 2024-04-16T15:40:44.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.960+0000 7f39bb7fe640 1 -- 172.21.15.71:0/267559709 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4037040 con 0x7f39d40c4350 2024-04-16T15:40:44.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.960+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d41ebea0 con 0x7f39d40c4350 2024-04-16T15:40:44.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.961+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4068d20 con 0x7f39d40c4350 2024-04-16T15:40:44.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.961+0000 7f39bb7fe640 1 -- 172.21.15.71:0/267559709 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39c4045070 con 0x7f39d40c4350 2024-04-16T15:40:44.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.961+0000 7f39bb7fe640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f39c006c8f0 0x7f39c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:44.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.962+0000 7f39d27fc640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f39c006c8f0 0x7f39c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:44.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.962+0000 7f39bb7fe640 1 -- 172.21.15.71:0/267559709 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f39c40addd0 con 0x7f39d40c4350 2024-04-16T15:40:44.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.962+0000 7f39d27fc640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f39c006c8f0 0x7f39c006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f39d414d3f0 tx=0x7f39c8007c60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:44.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:44.966+0000 7f39bb7fe640 1 -- 172.21.15.71:0/267559709 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39c407c070 con 0x7f39d40c4350 2024-04-16T15:40:45.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.177+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f39d414f2a0 con 0x7f39d40c4350 2024-04-16T15:40:45.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.177+0000 7f39bb7fe640 1 -- 172.21.15.71:0/267559709 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f39c407b970 con 0x7f39d40c4350 2024-04-16T15:40:45.177 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:45.177 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-16T15:40:45.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f39c006c8f0 msgr2=0x7f39c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f39c006c8f0 0x7f39c006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f39d414d3f0 tx=0x7f39c8007c60 comp rx=0 tx=0).stop 2024-04-16T15:40:45.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d40c4350 msgr2=0x7f39d41ec6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d40c4350 0x7f39d41ec6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f39c40097a0 tx=0x7f39c40046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 shutdown_connections 2024-04-16T15:40:45.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f39c006c8f0 0x7f39c006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d4159740 0x7f39d4148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39d4153570 0x7f39d414db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 --2- 172.21.15.71:0/267559709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d40c4350 0x7f39d41ec6a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 >> 172.21.15.71:0/267559709 conn(0x7f39d40c96e0 msgr2=0x7f39d40c5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:45.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 shutdown_connections 2024-04-16T15:40:45.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.179+0000 7f39d95f4640 1 -- 172.21.15.71:0/267559709 wait complete. 2024-04-16T15:40:45.190 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-16T15:40:45.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:45.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.293+0000 7f9bf53f0640 1 Processor -- start 2024-04-16T15:40:45.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.293+0000 7f9bf53f0640 1 -- start start 2024-04-16T15:40:45.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9bf53f0640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9bf0159520 0x7f9bf0159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9bf53f0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf00c78d0 0x7f9bf006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9bf53f0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9bf006e470 0x7f9bf00ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9bf53f0640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf00c9540 con 0x7f9bf0159520 2024-04-16T15:40:45.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9bf53f0640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf00ae4b0 con 0x7f9bf006e470 2024-04-16T15:40:45.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9bf53f0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf00c6940 con 0x7f9bf00c78d0 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9beeffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9bf0159520 0x7f9bf0159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9beeffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9bf0159520 0x7f9bf0159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47104/0 (socket says 172.21.15.71:47104) 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9beeffd640 1 -- 172.21.15.71:0/1470720973 learned_addr learned my addr 172.21.15.71:0/1470720973 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9be67fc640 1 --2- 172.21.15.71:0/1470720973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf00c78d0 0x7f9bf006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9bef7fe640 1 --2- 172.21.15.71:0/1470720973 >> v2:172.21.15.122:3300/0 conn(0x7f9bf006e470 0x7f9bf00ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9beeffd640 1 -- 172.21.15.71:0/1470720973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf00c78d0 msgr2=0x7f9bf006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9beeffd640 1 --2- 172.21.15.71:0/1470720973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf00c78d0 0x7f9bf006df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9beeffd640 1 -- 172.21.15.71:0/1470720973 >> v2:172.21.15.122:3300/0 conn(0x7f9bf006e470 msgr2=0x7f9bf00ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9beeffd640 1 --2- 172.21.15.71:0/1470720973 >> v2:172.21.15.122:3300/0 conn(0x7f9bf006e470 0x7f9bf00ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9beeffd640 1 -- 172.21.15.71:0/1470720973 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf015a030 con 0x7f9bf0159520 2024-04-16T15:40:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.294+0000 7f9be67fc640 1 --2- 172.21.15.71:0/1470720973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf00c78d0 0x7f9bf006df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.295+0000 7f9beeffd640 1 --2- 172.21.15.71:0/1470720973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0159900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b1080 tx=0x7f9be002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875684dce501d540 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.295+0000 7f9bee7fc640 1 -- 172.21.15.71:0/1470720973 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0038070 con 0x7f9bf0159520 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.295+0000 7f9bee7fc640 1 -- 172.21.15.71:0/1470720973 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be002f050 con 0x7f9bf0159520 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.295+0000 7f9bee7fc640 1 -- 172.21.15.71:0/1470720973 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0031970 con 0x7f9bf0159520 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 -- 172.21.15.71:0/1470720973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 msgr2=0x7f9bf0159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/1470720973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0159900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b1080 tx=0x7f9be002b7f0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 -- 172.21.15.71:0/1470720973 shutdown_connections 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/1470720973 >> v2:172.21.15.122:3300/0 conn(0x7f9bf006e470 0x7f9bf00ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/1470720973 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf00c78d0 0x7f9bf006df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/1470720973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0159900 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 -- 172.21.15.71:0/1470720973 >> 172.21.15.71:0/1470720973 conn(0x7f9bf00cb820 msgr2=0x7f9bf00cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 -- 172.21.15.71:0/1470720973 shutdown_connections 2024-04-16T15:40:45.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.296+0000 7f9bf53f0640 1 -- 172.21.15.71:0/1470720973 wait complete. 2024-04-16T15:40:45.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.297+0000 7f9bf53f0640 1 Processor -- start 2024-04-16T15:40:45.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.297+0000 7f9bf53f0640 1 -- start start 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bf53f0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf006e470 0x7f9bf00c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bf53f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bf00c78d0 0x7f9bf0154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bf53f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bf53f0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00c9d50 con 0x7f9bf0159520 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bf53f0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00c9a70 con 0x7f9bf00c78d0 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bf53f0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf00b0cf0 con 0x7f9bf006e470 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47106/0 (socket says 172.21.15.71:47106) 2024-04-16T15:40:45.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 -- 172.21.15.71:0/3999065234 learned_addr learned my addr 172.21.15.71:0/3999065234 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9beeffd640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf006e470 0x7f9bf00c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9be67fc640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bf00c78d0 0x7f9bf0154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 -- 172.21.15.71:0/3999065234 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf006e470 msgr2=0x7f9bf00c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf006e470 0x7f9bf00c1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 -- 172.21.15.71:0/3999065234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bf00c78d0 msgr2=0x7f9bf0154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bf00c78d0 0x7f9bf0154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 -- 172.21.15.71:0/3999065234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf0155d80 con 0x7f9bf0159520 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9bef7fe640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0155040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007d10 tx=0x7f9be800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.298+0000 7f9be7fff640 1 -- 172.21.15.71:0/3999065234 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8014420 con 0x7f9bf0159520 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.299+0000 7f9be7fff640 1 -- 172.21.15.71:0/3999065234 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be8019070 con 0x7f9bf0159520 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.299+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf0156060 con 0x7f9bf0159520 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.299+0000 7f9be7fff640 1 -- 172.21.15.71:0/3999065234 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8014820 con 0x7f9bf0159520 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.299+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf01ecec0 con 0x7f9bf0159520 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.300+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf0074060 con 0x7f9bf0159520 2024-04-16T15:40:45.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.300+0000 7f9be7fff640 1 -- 172.21.15.71:0/3999065234 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9be80149c0 con 0x7f9bf0159520 2024-04-16T15:40:45.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.300+0000 7f9be7fff640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9bd006c8f0 0x7f9bd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.301+0000 7f9be7fff640 1 -- 172.21.15.71:0/3999065234 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9be808c5d0 con 0x7f9bf0159520 2024-04-16T15:40:45.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.301+0000 7f9beeffd640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9bd006c8f0 0x7f9bd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.301+0000 7f9beeffd640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9bd006c8f0 0x7f9bd006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9be0035000 tx=0x7f9be0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.305+0000 7f9be7fff640 1 -- 172.21.15.71:0/3999065234 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9be805a750 con 0x7f9bf0159520 2024-04-16T15:40:45.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.482+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bf0156c60 con 0x7f9bf0159520 2024-04-16T15:40:45.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.483+0000 7f9be7fff640 1 -- 172.21.15.71:0/3999065234 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f9be8089860 con 0x7f9bf0159520 2024-04-16T15:40:45.483 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:45.483 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9bd006c8f0 msgr2=0x7f9bd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9bd006c8f0 0x7f9bd006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9be0035000 tx=0x7f9be0008030 comp rx=0 tx=0).stop 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 msgr2=0x7f9bf0155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0155040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9be8007d10 tx=0x7f9be800e4b0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 shutdown_connections 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9bd006c8f0 0x7f9bd006edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bf0159520 0x7f9bf0155040 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bf00c78d0 0x7f9bf0154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 --2- 172.21.15.71:0/3999065234 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bf006e470 0x7f9bf00c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 >> 172.21.15.71:0/3999065234 conn(0x7f9bf00cb820 msgr2=0x7f9bf00cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 shutdown_connections 2024-04-16T15:40:45.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.485+0000 7f9bf53f0640 1 -- 172.21.15.71:0/3999065234 wait complete. 2024-04-16T15:40:45.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:40:45.496 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:45.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.602+0000 7f8e1a7e9640 1 Processor -- start 2024-04-16T15:40:45.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.602+0000 7f8e1a7e9640 1 -- start start 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e1a7e9640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f8e14151470 0x7f8e140c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e1a7e9640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e140c5710 con 0x7f8e140c3cb0 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e1a7e9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e141544f0 con 0x7f8e14151090 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e1a7e9640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f8e14154ab0 con 0x7f8e14151470 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e18d86640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f8e14151470 0x7f8e140c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e137fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f8e140c3cb0 0x7f8e140c78a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34290/0 (socket says 172.21.15.71:34290) 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e18d86640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f8e14151470 0x7f8e140c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47112/0 (socket says 172.21.15.71:47112) 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e137fe640 1 -- 172.21.15.71:0/2504993622 learned_addr learned my addr 172.21.15.71:0/2504993622 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e18d86640 1 -- 172.21.15.71:0/2504993622 >> v1:172.21.15.122:6789/0 conn(0x7f8e14151090 legacy=0x7f8e14071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e18d86640 1 -- 172.21.15.71:0/2504993622 >> v1:172.21.15.71:6789/0 conn(0x7f8e140c3cb0 legacy=0x7f8e140c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.603+0000 7f8e18d86640 1 -- 172.21.15.71:0/2504993622 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e141547d0 con 0x7f8e14151470 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.604+0000 7f8e18d86640 1 --2- 172.21.15.71:0/2504993622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151470 0x7f8e140c7fb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e140c58e0 tx=0x7f8e0c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27519a661fd1668 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.604+0000 7f8e12ffd640 1 -- 172.21.15.71:0/2504993622 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c039070 con 0x7f8e14151470 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.604+0000 7f8e12ffd640 1 -- 172.21.15.71:0/2504993622 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0c02cc50 con 0x7f8e14151470 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.604+0000 7f8e12ffd640 1 -- 172.21.15.71:0/2504993622 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c034440 con 0x7f8e14151470 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.605+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/2504993622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151470 msgr2=0x7f8e140c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.605+0000 7f8e1a7e9640 1 --2- 172.21.15.71:0/2504993622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151470 0x7f8e140c7fb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e140c58e0 tx=0x7f8e0c02bea0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.605+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/2504993622 shutdown_connections 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.605+0000 7f8e1a7e9640 1 --2- 172.21.15.71:0/2504993622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151470 0x7f8e140c7fb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.605+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/2504993622 >> 172.21.15.71:0/2504993622 conn(0x7f8e140c5db0 msgr2=0x7f8e140c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.605+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/2504993622 shutdown_connections 2024-04-16T15:40:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.605+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/2504993622 wait complete. 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.606+0000 7f8e1a7e9640 1 Processor -- start 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.606+0000 7f8e1a7e9640 1 -- start start 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e1a7e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e140c3cb0 0x7f8e1414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e1a7e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151090 0x7f8e141534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e1a7e9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e14151470 0x7f8e1414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e1a7e9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8e140c84f0 con 0x7f8e14151090 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e1a7e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e140722d0 con 0x7f8e140c3cb0 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e1a7e9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8e14072450 con 0x7f8e14151470 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e137fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151090 0x7f8e141534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e137fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151090 0x7f8e141534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47120/0 (socket says 172.21.15.71:47120) 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e137fe640 1 -- 172.21.15.71:0/3919999440 learned_addr learned my addr 172.21.15.71:0/3919999440 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e13fff640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e140c3cb0 0x7f8e1414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e18d86640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e14151470 0x7f8e1414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e137fe640 1 -- 172.21.15.71:0/3919999440 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e14151470 msgr2=0x7f8e1414c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e137fe640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e14151470 0x7f8e1414c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e137fe640 1 -- 172.21.15.71:0/3919999440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e140c3cb0 msgr2=0x7f8e1414ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e137fe640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e140c3cb0 0x7f8e1414ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.607+0000 7f8e137fe640 1 -- 172.21.15.71:0/3919999440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e140aca00 con 0x7f8e14151090 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.608+0000 7f8e137fe640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151090 0x7f8e141534b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800c3c0 tx=0x7f8e0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.608+0000 7f8e10ff9640 1 -- 172.21.15.71:0/3919999440 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08017070 con 0x7f8e14151090 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.608+0000 7f8e10ff9640 1 -- 172.21.15.71:0/3919999440 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e08005ce0 con 0x7f8e14151090 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.608+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e141e94c0 con 0x7f8e14151090 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.608+0000 7f8e10ff9640 1 -- 172.21.15.71:0/3919999440 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e080072d0 con 0x7f8e14151090 2024-04-16T15:40:45.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.608+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e14152cb0 con 0x7f8e14151090 2024-04-16T15:40:45.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.609+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e14068be0 con 0x7f8e14151090 2024-04-16T15:40:45.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.609+0000 7f8e10ff9640 1 -- 172.21.15.71:0/3919999440 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e0801a070 con 0x7f8e14151090 2024-04-16T15:40:45.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.610+0000 7f8e10ff9640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8e0006c920 0x7f8e0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.610+0000 7f8e13fff640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8e0006c920 0x7f8e0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.610+0000 7f8e10ff9640 1 -- 172.21.15.71:0/3919999440 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8e0808b020 con 0x7f8e14151090 2024-04-16T15:40:45.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.610+0000 7f8e13fff640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8e0006c920 0x7f8e0006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8e14157c70 tx=0x7f8e04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.614+0000 7f8e10ff9640 1 -- 172.21.15.71:0/3919999440 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e08058b50 con 0x7f8e14151090 2024-04-16T15:40:45.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.792+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e140ad010 con 0x7f8e14151090 2024-04-16T15:40:45.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.794+0000 7f8e10ff9640 1 -- 172.21.15.71:0/3919999440 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f8e0805ce50 con 0x7f8e14151090 2024-04-16T15:40:45.794 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:45.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8e0006c920 msgr2=0x7f8e0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8e0006c920 0x7f8e0006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8e14157c70 tx=0x7f8e04009040 comp rx=0 tx=0).stop 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151090 msgr2=0x7f8e141534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151090 0x7f8e141534b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800c3c0 tx=0x7f8e0800c880 comp rx=0 tx=0).stop 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 shutdown_connections 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8e0006c920 0x7f8e0006ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e14151470 0x7f8e1414c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e14151090 0x7f8e141534b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 --2- 172.21.15.71:0/3919999440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e140c3cb0 0x7f8e1414ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 >> 172.21.15.71:0/3919999440 conn(0x7f8e140c5db0 msgr2=0x7f8e140c78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 shutdown_connections 2024-04-16T15:40:45.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.796+0000 7f8e1a7e9640 1 -- 172.21.15.71:0/3919999440 wait complete. 2024-04-16T15:40:45.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:40:45.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.913+0000 7fe0d39e9640 1 Processor -- start 2024-04-16T15:40:45.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.913+0000 7fe0d39e9640 1 -- start start 2024-04-16T15:40:45.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.914+0000 7fe0d39e9640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe0cc06fac0 0x7fe0cc0c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.914+0000 7fe0d39e9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe0cc0ab290 0x7fe0cc15d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.914+0000 7fe0d39e9640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0cc0c27c0 con 0x7fe0cc155dc0 2024-04-16T15:40:45.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.914+0000 7fe0d39e9640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fe0cc073f20 con 0x7fe0cc06fac0 2024-04-16T15:40:45.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.914+0000 7fe0d39e9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe0cc156ab0 con 0x7fe0cc0ab290 2024-04-16T15:40:45.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.914+0000 7fe0d0f84640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe0cc06fac0 0x7fe0cc0c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.914+0000 7fe0d1785640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fe0cc155dc0 0x7fe0cc1561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34296/0 (socket says 172.21.15.71:34296) 2024-04-16T15:40:45.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.914+0000 7fe0d1785640 1 -- 172.21.15.71:0/1763428103 learned_addr learned my addr 172.21.15.71:0/1763428103 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0d0f84640 1 -- 172.21.15.71:0/1763428103 >> v2:172.21.15.122:3300/0 conn(0x7fe0cc0ab290 msgr2=0x7fe0cc15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0d1f86640 1 --2- 172.21.15.71:0/1763428103 >> v2:172.21.15.122:3300/0 conn(0x7fe0cc0ab290 0x7fe0cc15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0d0f84640 1 --2- 172.21.15.71:0/1763428103 >> v2:172.21.15.122:3300/0 conn(0x7fe0cc0ab290 0x7fe0cc15d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0c3fff640 1 -- 172.21.15.71:0/1763428103 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1479373210 0 0) 0x7fe0cc0c27c0 con 0x7fe0cc155dc0 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0d0f84640 1 -- 172.21.15.71:0/1763428103 >> v1:172.21.15.71:6789/0 conn(0x7fe0cc155dc0 legacy=0x7fe0cc1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0d0f84640 1 -- 172.21.15.71:0/1763428103 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc0c2aa0 con 0x7fe0cc06fac0 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0d1f86640 1 --2- 172.21.15.71:0/1763428103 >> v2:172.21.15.122:3300/0 conn(0x7fe0cc0ab290 0x7fe0cc15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0d0f84640 1 --2- 172.21.15.71:0/1763428103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc06fac0 0x7fe0cc0c32a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc073bd0 tx=0x7fe0bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ddae21e2302452 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.915+0000 7fe0c3fff640 1 -- 172.21.15.71:0/1763428103 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc035020 con 0x7fe0cc06fac0 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0c3fff640 1 -- 172.21.15.71:0/1763428103 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0bc031830 con 0x7fe0cc06fac0 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0c3fff640 1 -- 172.21.15.71:0/1763428103 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc031c00 con 0x7fe0cc06fac0 2024-04-16T15:40:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0d39e9640 1 -- 172.21.15.71:0/1763428103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc06fac0 msgr2=0x7fe0cc0c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/1763428103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc06fac0 0x7fe0cc0c32a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc073bd0 tx=0x7fe0bc02b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0d39e9640 1 -- 172.21.15.71:0/1763428103 shutdown_connections 2024-04-16T15:40:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/1763428103 >> v2:172.21.15.122:3300/0 conn(0x7fe0cc0ab290 0x7fe0cc15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/1763428103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc06fac0 0x7fe0cc0c32a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0d39e9640 1 -- 172.21.15.71:0/1763428103 >> 172.21.15.71:0/1763428103 conn(0x7fe0cc0c3e10 msgr2=0x7fe0cc0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0d39e9640 1 -- 172.21.15.71:0/1763428103 shutdown_connections 2024-04-16T15:40:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.916+0000 7fe0d39e9640 1 -- 172.21.15.71:0/1763428103 wait complete. 2024-04-16T15:40:45.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.917+0000 7fe0d39e9640 1 Processor -- start 2024-04-16T15:40:45.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.917+0000 7fe0d39e9640 1 -- start start 2024-04-16T15:40:45.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d39e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc0ab290 0x7fe0cc1e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d39e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0cc155dc0 0x7fe0cc1ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d39e9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe0cc147300 0x7fe0cc1e7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d39e9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc06bd70 con 0x7fe0cc0ab290 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d39e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc0c18b0 con 0x7fe0cc155dc0 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d39e9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc06bef0 con 0x7fe0cc147300 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc0ab290 0x7fe0cc1e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc0ab290 0x7fe0cc1e9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47132/0 (socket says 172.21.15.71:47132) 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 -- 172.21.15.71:0/765828604 learned_addr learned my addr 172.21.15.71:0/765828604 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d0f84640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0cc155dc0 0x7fe0cc1ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1f86640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe0cc147300 0x7fe0cc1e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 -- 172.21.15.71:0/765828604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe0cc147300 msgr2=0x7fe0cc1e7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe0cc147300 0x7fe0cc1e7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 -- 172.21.15.71:0/765828604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0cc155dc0 msgr2=0x7fe0cc1ed3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0cc155dc0 0x7fe0cc1ed3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 -- 172.21.15.71:0/765828604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc072f60 con 0x7fe0cc0ab290 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.918+0000 7fe0d1785640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc0ab290 0x7fe0cc1e9fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c7c0 tx=0x7fe0c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.919+0000 7fe0c1ffb640 1 -- 172.21.15.71:0/765828604 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8018070 con 0x7fe0cc0ab290 2024-04-16T15:40:45.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.919+0000 7fe0c1ffb640 1 -- 172.21.15.71:0/765828604 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c800a640 con 0x7fe0cc0ab290 2024-04-16T15:40:45.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.919+0000 7fe0c1ffb640 1 -- 172.21.15.71:0/765828604 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800aa30 con 0x7fe0cc0ab290 2024-04-16T15:40:45.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.919+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0cc073240 con 0x7fe0cc0ab290 2024-04-16T15:40:45.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.919+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0cc075390 con 0x7fe0cc0ab290 2024-04-16T15:40:45.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.920+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0cc068be0 con 0x7fe0cc0ab290 2024-04-16T15:40:45.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.920+0000 7fe0c1ffb640 1 -- 172.21.15.71:0/765828604 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0c801b070 con 0x7fe0cc0ab290 2024-04-16T15:40:45.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.920+0000 7fe0c1ffb640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0b006c8f0 0x7fe0b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:45.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.921+0000 7fe0c1ffb640 1 -- 172.21.15.71:0/765828604 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe0c808b3a0 con 0x7fe0cc0ab290 2024-04-16T15:40:45.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.921+0000 7fe0d0f84640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0b006c8f0 0x7fe0b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:45.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.921+0000 7fe0d0f84640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0b006c8f0 0x7fe0b006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc036000 tx=0x7fe0bc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:45.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:45.925+0000 7fe0c1ffb640 1 -- 172.21.15.71:0/765828604 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe0c8058ed0 con 0x7fe0cc0ab290 2024-04-16T15:40:46.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.101+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0cc06bb50 con 0x7fe0cc0ab290 2024-04-16T15:40:46.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.101+0000 7fe0c1ffb640 1 -- 172.21.15.71:0/765828604 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fe0c805d1d0 con 0x7fe0cc0ab290 2024-04-16T15:40:46.101 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0b006c8f0 msgr2=0x7fe0b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0b006c8f0 0x7fe0b006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc036000 tx=0x7fe0bc03d040 comp rx=0 tx=0).stop 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc0ab290 msgr2=0x7fe0cc1e9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc0ab290 0x7fe0cc1e9fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c7c0 tx=0x7fe0c800cc80 comp rx=0 tx=0).stop 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 shutdown_connections 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0b006c8f0 0x7fe0b006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe0cc147300 0x7fe0cc1e7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0cc155dc0 0x7fe0cc1ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 --2- 172.21.15.71:0/765828604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0cc0ab290 0x7fe0cc1e9fc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 >> 172.21.15.71:0/765828604 conn(0x7fe0cc0c3e10 msgr2=0x7fe0cc147ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 shutdown_connections 2024-04-16T15:40:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.103+0000 7fe0d39e9640 1 -- 172.21.15.71:0/765828604 wait complete. 2024-04-16T15:40:46.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:40:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.219+0000 7efcdf9fa640 1 Processor -- start 2024-04-16T15:40:46.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.219+0000 7efcdf9fa640 1 -- start start 2024-04-16T15:40:46.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.220+0000 7efcdf9fa640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 0x7efcd80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.220+0000 7efcdf9fa640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd8156c70 con 0x7efcd80ca8f0 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.220+0000 7efcdf9fa640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd80c5ec0 con 0x7efcd8153430 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.220+0000 7efcdf9fa640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efcd80c67f0 con 0x7efcd8156770 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.220+0000 7efcddf97640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7efcd80ca8f0 0x7efcd8160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34312/0 (socket says 172.21.15.71:34312) 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.220+0000 7efcddf97640 1 -- 172.21.15.71:0/1133742478 learned_addr learned my addr 172.21.15.71:0/1133742478 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcdcf95640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 0x7efcd80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365112653 0 0) 0x7efcd8156c70 con 0x7efcd80ca8f0 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcc4003520 con 0x7efcd80ca8f0 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4026871259 0 0) 0x7efcc4003520 con 0x7efcd80ca8f0 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 msgr2=0x7efcd80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 0x7efcd80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 >> v1:172.21.15.122:6789/0 conn(0x7efcd8153430 legacy=0x7efcd81571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd80c61a0 con 0x7efcd80ca8f0 2024-04-16T15:40:46.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7efcd0001670 con 0x7efcd80ca8f0 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 >> v1:172.21.15.71:6789/0 conn(0x7efcd80ca8f0 legacy=0x7efcd8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc40039e0 0x7efcc4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc4004390 0x7efcc4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 0x7efcc4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efcd80c67f0 con 0x7efcc40039e0 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcc400bad0 con 0x7efcc4004390 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efcc400bcd0 con 0x7efcd8156770 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcdd796640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc40039e0 0x7efcc4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcdcf95640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc4004390 0x7efcc4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcddf97640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 0x7efcc4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcdd796640 1 -- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 msgr2=0x7efcc4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcdd796640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 0x7efcc4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcdd796640 1 -- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc4004390 msgr2=0x7efcc4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcdd796640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc4004390 0x7efcc4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcdd796640 1 -- 172.21.15.71:0/1133742478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc4003520 con 0x7efcc40039e0 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.221+0000 7efcddf97640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd8156770 0x7efcc4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.222+0000 7efcdd796640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc40039e0 0x7efcc4003dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efcd4001f60 tx=0x7efcc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d55e587cf53c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.222+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4039070 con 0x7efcc40039e0 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.222+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd402c730 con 0x7efcc40039e0 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.222+0000 7efccffff640 1 -- 172.21.15.71:0/1133742478 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4001b30 con 0x7efcc40039e0 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.223+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1133742478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc40039e0 msgr2=0x7efcc4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.223+0000 7efcdf9fa640 1 --2- 172.21.15.71:0/1133742478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcc40039e0 0x7efcc4003dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efcd4001f60 tx=0x7efcc4005830 comp rx=0 tx=0).stop 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.223+0000 7efcddf97640 1 -- 172.21.15.71:0/1133742478 reap_dead start 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.223+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1133742478 shutdown_connections 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.223+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1133742478 >> 172.21.15.71:0/1133742478 conn(0x7efcd80c7f20 msgr2=0x7efcd80c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.223+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1133742478 shutdown_connections 2024-04-16T15:40:46.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.223+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1133742478 wait complete. 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.224+0000 7efcdf9fa640 1 Processor -- start 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.224+0000 7efcdf9fa640 1 -- start start 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdf9fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc40039e0 0x7efcd806dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdf9fa640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd806e780 0x7efcd80c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdf9fa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcd80c2ab0 0x7efcd81ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdf9fa640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8153810 con 0x7efcd80c2ab0 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdf9fa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcd81578f0 con 0x7efcc40039e0 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdf9fa640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efcd8153990 con 0x7efcd806e780 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcd80c2ab0 0x7efcd81ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcd80c2ab0 0x7efcd81ebae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47154/0 (socket says 172.21.15.71:47154) 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 -- 172.21.15.71:0/1598336894 learned_addr learned my addr 172.21.15.71:0/1598336894 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdcf95640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd806e780 0x7efcd80c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdd796640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc40039e0 0x7efcd806dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 -- 172.21.15.71:0/1598336894 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd806e780 msgr2=0x7efcd80c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd806e780 0x7efcd80c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 -- 172.21.15.71:0/1598336894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc40039e0 msgr2=0x7efcd806dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc40039e0 0x7efcd806dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 -- 172.21.15.71:0/1598336894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd8146a20 con 0x7efcd80c2ab0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcddf97640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcd80c2ab0 0x7efcd81ebae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efcd808c960 tx=0x7efcd000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efccdffb640 1 -- 172.21.15.71:0/1598336894 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0010510 con 0x7efcd80c2ab0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd81559a0 con 0x7efcd80c2ab0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efccdffb640 1 -- 172.21.15.71:0/1598336894 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd00106d0 con 0x7efcd80c2ab0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.225+0000 7efccdffb640 1 -- 172.21.15.71:0/1598336894 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0010ac0 con 0x7efcd80c2ab0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.226+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd806e2c0 con 0x7efcd80c2ab0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.226+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd8073f50 con 0x7efcd80c2ab0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.227+0000 7efccdffb640 1 -- 172.21.15.71:0/1598336894 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcd0010c60 con 0x7efcd80c2ab0 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.227+0000 7efccdffb640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efcbc06c920 0x7efcbc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.227+0000 7efcdd796640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efcbc06c920 0x7efcbc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.227+0000 7efccdffb640 1 -- 172.21.15.71:0/1598336894 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efcd008b750 con 0x7efcd80c2ab0 2024-04-16T15:40:46.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.228+0000 7efcdd796640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efcbc06c920 0x7efcbc06ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcd4036000 tx=0x7efcd4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.232+0000 7efccdffb640 1 -- 172.21.15.71:0/1598336894 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcd00597c0 con 0x7efcd80c2ab0 2024-04-16T15:40:46.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.406+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efcd80aac60 con 0x7efcd80c2ab0 2024-04-16T15:40:46.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.407+0000 7efccdffb640 1 -- 172.21.15.71:0/1598336894 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7efcd80aac60 con 0x7efcd80c2ab0 2024-04-16T15:40:46.406 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:40:46.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efcbc06c920 msgr2=0x7efcbc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efcbc06c920 0x7efcbc06ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcd4036000 tx=0x7efcd4009030 comp rx=0 tx=0).stop 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcd80c2ab0 msgr2=0x7efcd81ebae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcd80c2ab0 0x7efcd81ebae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efcd808c960 tx=0x7efcd000ead0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 shutdown_connections 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efcbc06c920 0x7efcbc06ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efcd80c2ab0 0x7efcd81ebae0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efcd806e780 0x7efcd80c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 --2- 172.21.15.71:0/1598336894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcc40039e0 0x7efcd806dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.408+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 >> 172.21.15.71:0/1598336894 conn(0x7efcd80c7f20 msgr2=0x7efcd8151130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.409+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 shutdown_connections 2024-04-16T15:40:46.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.409+0000 7efcdf9fa640 1 -- 172.21.15.71:0/1598336894 wait complete. 2024-04-16T15:40:46.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:40:46.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.526+0000 7f9e86693640 1 Processor -- start 2024-04-16T15:40:46.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.526+0000 7f9e86693640 1 -- start start 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e86693640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e80150ec0 0x7f9e800721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e86693640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9e80063170 0x7f9e800c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e86693640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9e800c1d20 0x7f9e8006c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e86693640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f9e80154550 con 0x7f9e800c1d20 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e86693640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9e801547f0 con 0x7f9e80063170 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e86693640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9e800c3580 con 0x7f9e80150ec0 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e84c30640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9e800c1d20 0x7f9e8006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e84c30640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9e800c1d20 0x7f9e8006c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47168/0 (socket says 172.21.15.71:47168) 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e84c30640 1 -- 172.21.15.71:0/2088606270 learned_addr learned my addr 172.21.15.71:0/2088606270 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e7f7fe640 1 --2- 172.21.15.71:0/2088606270 >> v2:172.21.15.122:3300/0 conn(0x7f9e80063170 0x7f9e800c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e7ffff640 1 --2- 172.21.15.71:0/2088606270 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e80150ec0 0x7f9e800721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e84c30640 1 -- 172.21.15.71:0/2088606270 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e80150ec0 msgr2=0x7f9e800721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e84c30640 1 --2- 172.21.15.71:0/2088606270 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e80150ec0 0x7f9e800721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e84c30640 1 -- 172.21.15.71:0/2088606270 >> v2:172.21.15.122:3300/0 conn(0x7f9e80063170 msgr2=0x7f9e800c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e84c30640 1 --2- 172.21.15.71:0/2088606270 >> v2:172.21.15.122:3300/0 conn(0x7f9e80063170 0x7f9e800c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e84c30640 1 -- 172.21.15.71:0/2088606270 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e800c3860 con 0x7f9e800c1d20 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.527+0000 7f9e7f7fe640 1 --2- 172.21.15.71:0/2088606270 >> v2:172.21.15.122:3300/0 conn(0x7f9e80063170 0x7f9e800c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.528+0000 7f9e84c30640 1 --2- 172.21.15.71:0/2088606270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800c1d20 0x7f9e8006c940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e800add10 tx=0x7f9e7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4657bfab3ca5fda server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.528+0000 7f9e7effd640 1 -- 172.21.15.71:0/2088606270 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7802ca70 con 0x7f9e800c1d20 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.528+0000 7f9e7effd640 1 -- 172.21.15.71:0/2088606270 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7802cc10 con 0x7f9e800c1d20 2024-04-16T15:40:46.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.528+0000 7f9e7effd640 1 -- 172.21.15.71:0/2088606270 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e780343f0 con 0x7f9e800c1d20 2024-04-16T15:40:46.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 -- 172.21.15.71:0/2088606270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800c1d20 msgr2=0x7f9e8006c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 --2- 172.21.15.71:0/2088606270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800c1d20 0x7f9e8006c940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e800add10 tx=0x7f9e7802bed0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 -- 172.21.15.71:0/2088606270 shutdown_connections 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 --2- 172.21.15.71:0/2088606270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800c1d20 0x7f9e8006c940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 --2- 172.21.15.71:0/2088606270 >> v2:172.21.15.122:3300/0 conn(0x7f9e80063170 0x7f9e800c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 --2- 172.21.15.71:0/2088606270 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e80150ec0 0x7f9e800721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 -- 172.21.15.71:0/2088606270 >> 172.21.15.71:0/2088606270 conn(0x7f9e800c45a0 msgr2=0x7f9e800c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 -- 172.21.15.71:0/2088606270 shutdown_connections 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.529+0000 7f9e86693640 1 -- 172.21.15.71:0/2088606270 wait complete. 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.530+0000 7f9e86693640 1 Processor -- start 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.530+0000 7f9e86693640 1 -- start start 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e86693640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e800c1d20 0x7f9e80156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e86693640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e80150ec0 0x7f9e8014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e86693640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800707a0 0x7f9e801eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e86693640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801512a0 con 0x7f9e800707a0 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e86693640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9e800c3230 con 0x7f9e80150ec0 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e86693640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9e80073b50 con 0x7f9e800c1d20 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800707a0 0x7f9e801eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800707a0 0x7f9e801eb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47170/0 (socket says 172.21.15.71:47170) 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 -- 172.21.15.71:0/151839805 learned_addr learned my addr 172.21.15.71:0/151839805 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e7ffff640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e800c1d20 0x7f9e80156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e7f7fe640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e80150ec0 0x7f9e8014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 -- 172.21.15.71:0/151839805 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e800c1d20 msgr2=0x7f9e80156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e800c1d20 0x7f9e80156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 -- 172.21.15.71:0/151839805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e80150ec0 msgr2=0x7f9e8014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e80150ec0 0x7f9e8014f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 -- 172.21.15.71:0/151839805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e801571e0 con 0x7f9e800707a0 2024-04-16T15:40:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e84c30640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800707a0 0x7f9e801eb930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e78036040 tx=0x7f9e78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e7cff9640 1 -- 172.21.15.71:0/151839805 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78043020 con 0x7f9e800707a0 2024-04-16T15:40:46.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e7cff9640 1 -- 172.21.15.71:0/151839805 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7803e450 con 0x7f9e800707a0 2024-04-16T15:40:46.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e7cff9640 1 -- 172.21.15.71:0/151839805 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78009510 con 0x7f9e800707a0 2024-04-16T15:40:46.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e800a78c0 con 0x7f9e800707a0 2024-04-16T15:40:46.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.531+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e800c1630 con 0x7f9e800707a0 2024-04-16T15:40:46.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.532+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e80068be0 con 0x7f9e800707a0 2024-04-16T15:40:46.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.533+0000 7f9e7cff9640 1 -- 172.21.15.71:0/151839805 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e78046020 con 0x7f9e800707a0 2024-04-16T15:40:46.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.533+0000 7f9e7cff9640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9e6c06c8f0 0x7f9e6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.534+0000 7f9e7ffff640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9e6c06c8f0 0x7f9e6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.534+0000 7f9e7cff9640 1 -- 172.21.15.71:0/151839805 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9e780ae530 con 0x7f9e800707a0 2024-04-16T15:40:46.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.534+0000 7f9e7ffff640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9e6c06c8f0 0x7f9e6c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006fd0 tx=0x7f9e70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.538+0000 7f9e7cff9640 1 -- 172.21.15.71:0/151839805 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e7804d030 con 0x7f9e800707a0 2024-04-16T15:40:46.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.712+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9e800a74a0 con 0x7f9e800707a0 2024-04-16T15:40:46.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.712+0000 7f9e7cff9640 1 -- 172.21.15.71:0/151839805 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f9e7807c060 con 0x7f9e800707a0 2024-04-16T15:40:46.712 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:46.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.714+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9e6c06c8f0 msgr2=0x7f9e6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.714+0000 7f9e86693640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9e6c06c8f0 0x7f9e6c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006fd0 tx=0x7f9e70008040 comp rx=0 tx=0).stop 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.714+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800707a0 msgr2=0x7f9e801eb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.714+0000 7f9e86693640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800707a0 0x7f9e801eb930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e78036040 tx=0x7f9e78004a40 comp rx=0 tx=0).stop 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.715+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 shutdown_connections 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.715+0000 7f9e86693640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9e6c06c8f0 0x7f9e6c06edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.715+0000 7f9e86693640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e800707a0 0x7f9e801eb930 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.715+0000 7f9e86693640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e80150ec0 0x7f9e8014f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.715+0000 7f9e86693640 1 --2- 172.21.15.71:0/151839805 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9e800c1d20 0x7f9e80156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.715+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 >> 172.21.15.71:0/151839805 conn(0x7f9e800c45a0 msgr2=0x7f9e800ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:46.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.715+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 shutdown_connections 2024-04-16T15:40:46.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.715+0000 7f9e86693640 1 -- 172.21.15.71:0/151839805 wait complete. 2024-04-16T15:40:46.726 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:40:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.829+0000 7f8460faa640 1 Processor -- start 2024-04-16T15:40:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.829+0000 7f8460faa640 1 -- start start 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.830+0000 7f8460faa640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f845c0c41f0 0x7f845c0c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.830+0000 7f8460faa640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f845c150490 con 0x7f845c14fbd0 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.830+0000 7f8460faa640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f845c0aac10 con 0x7f845c150040 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.830+0000 7f8460faa640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f845c0aa9c0 con 0x7f845c0c41f0 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.830+0000 7f8459d9b640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f845c14fbd0 0x7f845c152b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34318/0 (socket says 172.21.15.71:34318) 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.830+0000 7f8459d9b640 1 -- 172.21.15.71:0/1312884024 learned_addr learned my addr 172.21.15.71:0/1312884024 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845a59c640 1 --2- 172.21.15.71:0/1312884024 >> v2:172.21.15.122:3300/0 conn(0x7f845c0c41f0 0x7f845c0c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160650475 0 0) 0x7f845c150490 con 0x7f845c14fbd0 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8444005730 con 0x7f845c14fbd0 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2461766737 0 0) 0x7f8444005730 con 0x7f845c14fbd0 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 >> v2:172.21.15.122:3300/0 conn(0x7f845c0c41f0 msgr2=0x7f845c0c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:46.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 --2- 172.21.15.71:0/1312884024 >> v2:172.21.15.122:3300/0 conn(0x7f845c0c41f0 0x7f845c0c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 >> v1:172.21.15.122:6789/0 conn(0x7f845c150040 legacy=0x7f845c153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c0a9640 con 0x7f845c14fbd0 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f8450001670 con 0x7f845c14fbd0 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 >> v1:172.21.15.71:6789/0 conn(0x7f845c14fbd0 legacy=0x7f845c152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8444005bc0 0x7f8444005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f8444006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8444007420 0x7f844400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f845c0aa9c0 con 0x7f8444006a10 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f844400e370 con 0x7f8444005bc0 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f844400e570 con 0x7f8444007420 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f8459d9b640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f8444006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845a59c640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8444005bc0 0x7f8444005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845ad9d640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8444007420 0x7f844400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f8459d9b640 1 -- 172.21.15.71:0/1312884024 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8444007420 msgr2=0x7f844400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f8459d9b640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8444007420 0x7f844400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f8459d9b640 1 -- 172.21.15.71:0/1312884024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8444005bc0 msgr2=0x7f8444005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f8459d9b640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8444005bc0 0x7f8444005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f8459d9b640 1 -- 172.21.15.71:0/1312884024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c0a9640 con 0x7f8444006a10 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.831+0000 7f845ad9d640 1 -- 172.21.15.71:0/1312884024 reap_dead start 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.832+0000 7f8459d9b640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f8444006e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8450001050 tx=0x7f8444007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9014f40ddef41562 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.832+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450007dd0 con 0x7f8444006a10 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.832+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8450030070 con 0x7f8444006a10 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.832+0000 7f845959a640 1 -- 172.21.15.71:0/1312884024 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845002ce10 con 0x7f8444006a10 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.833+0000 7f8460faa640 1 -- 172.21.15.71:0/1312884024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 msgr2=0x7f8444006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.833+0000 7f8460faa640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f8444006e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8450001050 tx=0x7f8444007a40 comp rx=0 tx=0).stop 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.833+0000 7f8460faa640 1 -- 172.21.15.71:0/1312884024 shutdown_connections 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.833+0000 7f8460faa640 1 --2- 172.21.15.71:0/1312884024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f8444006e90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.833+0000 7f8460faa640 1 -- 172.21.15.71:0/1312884024 >> 172.21.15.71:0/1312884024 conn(0x7f845c0c4e20 msgr2=0x7f845c0c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:46.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.833+0000 7f8460faa640 1 -- 172.21.15.71:0/1312884024 shutdown_connections 2024-04-16T15:40:46.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.833+0000 7f8460faa640 1 -- 172.21.15.71:0/1312884024 wait complete. 2024-04-16T15:40:46.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.834+0000 7f8460faa640 1 Processor -- start 2024-04-16T15:40:46.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.834+0000 7f8460faa640 1 -- start start 2024-04-16T15:40:46.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f8460faa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f845c14bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f8460faa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c155990 0x7f845c142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f8460faa640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f845c154660 0x7f845c1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f8460faa640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f845c0714c0 con 0x7f8444006a10 2024-04-16T15:40:46.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f8460faa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f845c071340 con 0x7f845c155990 2024-04-16T15:40:46.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f8460faa640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f845c070360 con 0x7f845c154660 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f845c14bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f845c14bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47184/0 (socket says 172.21.15.71:47184) 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 -- 172.21.15.71:0/817669920 learned_addr learned my addr 172.21.15.71:0/817669920 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f8459d9b640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c155990 0x7f845c142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845ad9d640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f845c154660 0x7f845c1430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 -- 172.21.15.71:0/817669920 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f845c154660 msgr2=0x7f845c1430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f845c154660 0x7f845c1430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 -- 172.21.15.71:0/817669920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c155990 msgr2=0x7f845c142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c155990 0x7f845c142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 -- 172.21.15.71:0/817669920 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c1e6da0 con 0x7f8444006a10 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f845a59c640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f845c14bc00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f844c011e60 tx=0x7f844c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.835+0000 7f844affd640 1 -- 172.21.15.71:0/817669920 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c015630 con 0x7f8444006a10 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.836+0000 7f844affd640 1 -- 172.21.15.71:0/817669920 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c0157d0 con 0x7f8444006a10 2024-04-16T15:40:46.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.836+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845c1e7060 con 0x7f8444006a10 2024-04-16T15:40:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.836+0000 7f844affd640 1 -- 172.21.15.71:0/817669920 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c015ba0 con 0x7f8444006a10 2024-04-16T15:40:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.836+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845c1e7620 con 0x7f8444006a10 2024-04-16T15:40:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.836+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845c068cf0 con 0x7f8444006a10 2024-04-16T15:40:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.837+0000 7f844affd640 1 -- 172.21.15.71:0/817669920 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f844c01d400 con 0x7f8444006a10 2024-04-16T15:40:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.837+0000 7f844affd640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.837+0000 7f844affd640 1 -- 172.21.15.71:0/817669920 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f844c08d050 con 0x7f8444006a10 2024-04-16T15:40:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.837+0000 7f8459d9b640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.838+0000 7f8459d9b640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f843c06c8f0 0x7f843c06edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8450034000 tx=0x7f8450009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:46.842+0000 7f844affd640 1 -- 172.21.15.71:0/817669920 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f844c019030 con 0x7f8444006a10 2024-04-16T15:40:47.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.055+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f845c070140 con 0x7f8444006a10 2024-04-16T15:40:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.055+0000 7f844affd640 1 -- 172.21.15.71:0/817669920 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f844c010040 con 0x7f8444006a10 2024-04-16T15:40:47.068 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:47.068 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:40.120873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:40:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-04-16T15:40:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f843c06c8f0 msgr2=0x7f843c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f843c06c8f0 0x7f843c06edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8450034000 tx=0x7f8450009030 comp rx=0 tx=0).stop 2024-04-16T15:40:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 msgr2=0x7f845c14bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f845c14bc00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f844c011e60 tx=0x7f844c00f580 comp rx=0 tx=0).stop 2024-04-16T15:40:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 shutdown_connections 2024-04-16T15:40:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f845c154660 0x7f845c1430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c155990 0x7f845c142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 --2- 172.21.15.71:0/817669920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444006a10 0x7f845c14bc00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.057+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 >> 172.21.15.71:0/817669920 conn(0x7f845c0c4e20 msgr2=0x7f845c0c7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.058+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 shutdown_connections 2024-04-16T15:40:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.058+0000 7f8460faa640 1 -- 172.21.15.71:0/817669920 wait complete. 2024-04-16T15:40:47.071 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:47.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.174+0000 7fe6fa1bd640 1 Processor -- start 2024-04-16T15:40:47.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.174+0000 7fe6fa1bd640 1 -- start start 2024-04-16T15:40:47.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6fa1bd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe6f40713a0 0x7fe6f4155050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6fa1bd640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f41549a0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6fa1bd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f40c7650 con 0x7fe6f40c8130 2024-04-16T15:40:47.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6fa1bd640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f40c7400 con 0x7fe6f40713a0 2024-04-16T15:40:47.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6f37fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe6f40713a0 0x7fe6f4155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6f3fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fe6f40c8ee0 0x7fe6f40accc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34332/0 (socket says 172.21.15.71:34332) 2024-04-16T15:40:47.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6f3fff640 1 -- 172.21.15.71:0/958810301 learned_addr learned my addr 172.21.15.71:0/958810301 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:47.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6f37fe640 1 -- 172.21.15.71:0/958810301 >> v1:172.21.15.122:6789/0 conn(0x7fe6f40c8130 legacy=0x7fe6f40ac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:47.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6f37fe640 1 -- 172.21.15.71:0/958810301 >> v1:172.21.15.71:6789/0 conn(0x7fe6f40c8ee0 legacy=0x7fe6f40accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6f37fe640 1 -- 172.21.15.71:0/958810301 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f40c7930 con 0x7fe6f40713a0 2024-04-16T15:40:47.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.175+0000 7fe6f27fc640 1 -- 172.21.15.71:0/958810301 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1661076776 0 0) 0x7fe6f41549a0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.176+0000 7fe6f37fe640 1 --2- 172.21.15.71:0/958810301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40713a0 0x7fe6f4155050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009d90 tx=0x7fe6e402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a8e421258f8dda server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.176+0000 7fe6f27fc640 1 -- 172.21.15.71:0/958810301 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4039070 con 0x7fe6f40713a0 2024-04-16T15:40:47.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.176+0000 7fe6f27fc640 1 -- 172.21.15.71:0/958810301 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e402cc60 con 0x7fe6f40713a0 2024-04-16T15:40:47.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.176+0000 7fe6f27fc640 1 -- 172.21.15.71:0/958810301 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4034660 con 0x7fe6f40713a0 2024-04-16T15:40:47.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.177+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/958810301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40713a0 msgr2=0x7fe6f4155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.177+0000 7fe6fa1bd640 1 --2- 172.21.15.71:0/958810301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40713a0 0x7fe6f4155050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009d90 tx=0x7fe6e402beb0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.177+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/958810301 shutdown_connections 2024-04-16T15:40:47.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.177+0000 7fe6fa1bd640 1 --2- 172.21.15.71:0/958810301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40713a0 0x7fe6f4155050 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.177+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/958810301 >> 172.21.15.71:0/958810301 conn(0x7fe6f40c96e0 msgr2=0x7fe6f40c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:47.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.177+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/958810301 shutdown_connections 2024-04-16T15:40:47.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.177+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/958810301 wait complete. 2024-04-16T15:40:47.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.178+0000 7fe6fa1bd640 1 Processor -- start 2024-04-16T15:40:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.178+0000 7fe6fa1bd640 1 -- start start 2024-04-16T15:40:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6fa1bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f40713a0 0x7fe6f40bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6fa1bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe6f40c8130 0x7fe6f414db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6fa1bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40c8ee0 0x7fe6f41e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6fa1bd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4154590 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6fa1bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40c7ca0 con 0x7fe6f40713a0 2024-04-16T15:40:47.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6fa1bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f4154710 con 0x7fe6f40c8130 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40c8ee0 0x7fe6f41e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40c8ee0 0x7fe6f41e66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47194/0 (socket says 172.21.15.71:47194) 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 -- 172.21.15.71:0/2877899418 learned_addr learned my addr 172.21.15.71:0/2877899418 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f37fe640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f40713a0 0x7fe6f40bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f2ffd640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe6f40c8130 0x7fe6f414db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 -- 172.21.15.71:0/2877899418 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe6f40c8130 msgr2=0x7fe6f414db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe6f40c8130 0x7fe6f414db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 -- 172.21.15.71:0/2877899418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f40713a0 msgr2=0x7fe6f40bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f40713a0 0x7fe6f40bf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 -- 172.21.15.71:0/2877899418 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f40c35d0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6f3fff640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40c8ee0 0x7fe6f41e66b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00c440 tx=0x7fe6ec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.179+0000 7fe6dbfff640 1 -- 172.21.15.71:0/2877899418 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0074e0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.180+0000 7fe6dbfff640 1 -- 172.21.15.71:0/2877899418 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ec0047e0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.180+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f41ea2a0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.180+0000 7fe6dbfff640 1 -- 172.21.15.71:0/2877899418 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec004bd0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.180+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f41e9fc0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.181+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f41e9db0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.181+0000 7fe6dbfff640 1 -- 172.21.15.71:0/2877899418 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6ec011070 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.182+0000 7fe6dbfff640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe6e006c920 0x7fe6e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.182+0000 7fe6dbfff640 1 -- 172.21.15.71:0/2877899418 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe6ec08b7a0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.182+0000 7fe6f37fe640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe6e006c920 0x7fe6e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.182+0000 7fe6f37fe640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe6e006c920 0x7fe6e006ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4036000 tx=0x7fe6e4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.186+0000 7fe6dbfff640 1 -- 172.21.15.71:0/2877899418 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6ec0592d0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.363+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6f40c3f00 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.364+0000 7fe6dbfff640 1 -- 172.21.15.71:0/2877899418 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7fe6ec05d5d0 con 0x7fe6f40c8ee0 2024-04-16T15:40:47.364 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:47.364 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe6e006c920 msgr2=0x7fe6e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe6e006c920 0x7fe6e006ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4036000 tx=0x7fe6e4030000 comp rx=0 tx=0).stop 2024-04-16T15:40:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40c8ee0 msgr2=0x7fe6f41e66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40c8ee0 0x7fe6f41e66b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00c440 tx=0x7fe6ec00c900 comp rx=0 tx=0).stop 2024-04-16T15:40:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 shutdown_connections 2024-04-16T15:40:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe6e006c920 0x7fe6e006ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f40c8ee0 0x7fe6f41e66b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe6f40c8130 0x7fe6f414db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 --2- 172.21.15.71:0/2877899418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f40713a0 0x7fe6f40bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 >> 172.21.15.71:0/2877899418 conn(0x7fe6f40c96e0 msgr2=0x7fe6f4070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 shutdown_connections 2024-04-16T15:40:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.366+0000 7fe6fa1bd640 1 -- 172.21.15.71:0/2877899418 wait complete. 2024-04-16T15:40:47.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:40:47.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:47.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.483+0000 7f593a382640 1 Processor -- start 2024-04-16T15:40:47.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.483+0000 7f593a382640 1 -- start start 2024-04-16T15:40:47.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f593a382640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f593413f260 0x7f59340aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f593a382640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 0x7f5934156170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f593a382640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59341592a0 con 0x7f59340aaec0 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f593a382640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f5934159580 con 0x7f593413f260 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f593a382640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5934159860 con 0x7f5934155d20 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f5933fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f593413f260 0x7f59340aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f59337fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f59340aaec0 0x7f593406da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:46480/0 (socket says 172.21.15.71:46480) 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f59337fe640 1 -- 172.21.15.71:0/748444180 learned_addr learned my addr 172.21.15.71:0/748444180 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.484+0000 7f593891f640 1 --2- 172.21.15.71:0/748444180 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 0x7f5934156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.485+0000 7f5933fff640 1 -- 172.21.15.71:0/748444180 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 msgr2=0x7f5934156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.485+0000 7f5933fff640 1 --2- 172.21.15.71:0/748444180 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 0x7f5934156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.485+0000 7f5933fff640 1 -- 172.21.15.71:0/748444180 >> v1:172.21.15.122:6789/0 conn(0x7f59340aaec0 legacy=0x7f593406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.485+0000 7f5933fff640 1 -- 172.21.15.71:0/748444180 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5934159b40 con 0x7f593413f260 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f5933fff640 1 --2- 172.21.15.71:0/748444180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593413f260 0x7f59340aa980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59240097a0 tx=0x7f592402b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6bf62cb82e41e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f5932ffd640 1 -- 172.21.15.71:0/748444180 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924038070 con 0x7f593413f260 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f5932ffd640 1 -- 172.21.15.71:0/748444180 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5924031700 con 0x7f593413f260 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f5932ffd640 1 -- 172.21.15.71:0/748444180 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924031ad0 con 0x7f593413f260 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f593a382640 1 -- 172.21.15.71:0/748444180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593413f260 msgr2=0x7f59340aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f593a382640 1 --2- 172.21.15.71:0/748444180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593413f260 0x7f59340aa980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59240097a0 tx=0x7f592402b7a0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f593a382640 1 -- 172.21.15.71:0/748444180 shutdown_connections 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f593a382640 1 --2- 172.21.15.71:0/748444180 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 0x7f5934156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f593a382640 1 --2- 172.21.15.71:0/748444180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593413f260 0x7f59340aa980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f593a382640 1 -- 172.21.15.71:0/748444180 >> 172.21.15.71:0/748444180 conn(0x7f59340729d0 msgr2=0x7f5934072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f593a382640 1 -- 172.21.15.71:0/748444180 shutdown_connections 2024-04-16T15:40:47.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.486+0000 7f593a382640 1 -- 172.21.15.71:0/748444180 wait complete. 2024-04-16T15:40:47.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.487+0000 7f593a382640 1 Processor -- start 2024-04-16T15:40:47.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f593a382640 1 -- start start 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f593a382640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59340aaec0 0x7f593413b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f593a382640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f593413f260 0x7f59341586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f593a382640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 0x7f5934137070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f593a382640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f59340c1ea0 con 0x7f593413f260 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f593a382640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59340c1d20 con 0x7f59340aaec0 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f593a382640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f59341566b0 con 0x7f5934155d20 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f59337fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f593413f260 0x7f59341586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f59337fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f593413f260 0x7f59341586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52696/0 (socket says 172.21.15.71:52696) 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.488+0000 7f59337fe640 1 -- 172.21.15.71:0/2663449909 learned_addr learned my addr 172.21.15.71:0/2663449909 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f5933fff640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59340aaec0 0x7f593413b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f593891f640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 0x7f5934137070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f59337fe640 1 -- 172.21.15.71:0/2663449909 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 msgr2=0x7f5934137070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f59337fe640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 0x7f5934137070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f59337fe640 1 -- 172.21.15.71:0/2663449909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59340aaec0 msgr2=0x7f593413b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f59337fe640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59340aaec0 0x7f593413b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f59337fe640 1 -- 172.21.15.71:0/2663449909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593413bc10 con 0x7f593413f260 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f59337fe640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f593413f260 0x7f59341586f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f592800c440 tx=0x7f5928001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f5930ff9640 1 -- 172.21.15.71:0/2663449909 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5928017070 con 0x7f593413f260 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f5930ff9640 1 -- 172.21.15.71:0/2663449909 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5928005d00 con 0x7f593413f260 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59341ef6c0 con 0x7f593413f260 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f5930ff9640 1 -- 172.21.15.71:0/2663449909 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5928007310 con 0x7f593413f260 2024-04-16T15:40:47.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.489+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59340c7ab0 con 0x7f593413f260 2024-04-16T15:40:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.490+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5934068be0 con 0x7f593413f260 2024-04-16T15:40:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.491+0000 7f5930ff9640 1 -- 172.21.15.71:0/2663449909 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f592801a070 con 0x7f593413f260 2024-04-16T15:40:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.491+0000 7f5930ff9640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f592006c8f0 0x7f592006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.491+0000 7f5933fff640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f592006c8f0 0x7f592006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.491+0000 7f5930ff9640 1 -- 172.21.15.71:0/2663449909 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f592808b100 con 0x7f593413f260 2024-04-16T15:40:47.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.492+0000 7f5933fff640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f592006c8f0 0x7f592006edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5924035000 tx=0x7f5924008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.496+0000 7f5930ff9640 1 -- 172.21.15.71:0/2663449909 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5928058c30 con 0x7f593413f260 2024-04-16T15:40:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.671+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59340c1970 con 0x7f593413f260 2024-04-16T15:40:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.672+0000 7f5930ff9640 1 -- 172.21.15.71:0/2663449909 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f592805cf30 con 0x7f593413f260 2024-04-16T15:40:47.672 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f592006c8f0 msgr2=0x7f592006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f592006c8f0 0x7f592006edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5924035000 tx=0x7f5924008030 comp rx=0 tx=0).stop 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f593413f260 msgr2=0x7f59341586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f593413f260 0x7f59341586f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f592800c440 tx=0x7f5928001e70 comp rx=0 tx=0).stop 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 shutdown_connections 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f592006c8f0 0x7f592006edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5934155d20 0x7f5934137070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f593413f260 0x7f59341586f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 --2- 172.21.15.71:0/2663449909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59340aaec0 0x7f593413b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 >> 172.21.15.71:0/2663449909 conn(0x7f59340729d0 msgr2=0x7f5934072db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:47.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 shutdown_connections 2024-04-16T15:40:47.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.674+0000 7f593a382640 1 -- 172.21.15.71:0/2663449909 wait complete. 2024-04-16T15:40:47.685 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:40:47.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.791+0000 7f4c11f96640 1 Processor -- start 2024-04-16T15:40:47.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.791+0000 7f4c11f96640 1 -- start start 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c11f96640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f4c0c0c4350 0x7f4c0c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c11f96640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0c0c82d0 con 0x7f4c0c159740 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c11f96640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0c155d70 con 0x7f4c0c153570 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c11f96640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f4c0c15a2e0 con 0x7f4c0c0c4350 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c03fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f4c0c0c4350 0x7f4c0c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c0bfff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f4c0c159740 0x7f4c0c159bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34776/0 (socket says 172.21.15.71:34776) 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c0bfff640 1 -- 172.21.15.71:0/3361051307 learned_addr learned my addr 172.21.15.71:0/3361051307 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c03fff640 1 -- 172.21.15.71:0/3361051307 >> v1:172.21.15.122:6789/0 conn(0x7f4c0c153570 legacy=0x7f4c0c0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c03fff640 1 -- 172.21.15.71:0/3361051307 >> v1:172.21.15.71:6789/0 conn(0x7f4c0c159740 legacy=0x7f4c0c159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c0affd640 1 -- 172.21.15.71:0/3361051307 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318424183 0 0) 0x7f4c0c0c82d0 con 0x7f4c0c159740 2024-04-16T15:40:47.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.792+0000 7f4c03fff640 1 -- 172.21.15.71:0/3361051307 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c0c8d50 con 0x7f4c0c0c4350 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.793+0000 7f4c03fff640 1 --2- 172.21.15.71:0/3361051307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c0c4350 0x7f4c0c0c52b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40097a0 tx=0x7f4bf402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f1effce5bfda server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.793+0000 7f4c0affd640 1 -- 172.21.15.71:0/3361051307 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4038070 con 0x7f4c0c0c4350 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.793+0000 7f4c0affd640 1 -- 172.21.15.71:0/3361051307 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf4031700 con 0x7f4c0c0c4350 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.793+0000 7f4c0affd640 1 -- 172.21.15.71:0/3361051307 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4031ad0 con 0x7f4c0c0c4350 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.794+0000 7f4c11f96640 1 -- 172.21.15.71:0/3361051307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c0c4350 msgr2=0x7f4c0c0c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.794+0000 7f4c11f96640 1 --2- 172.21.15.71:0/3361051307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c0c4350 0x7f4c0c0c52b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40097a0 tx=0x7f4bf402b7a0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.794+0000 7f4c11f96640 1 -- 172.21.15.71:0/3361051307 shutdown_connections 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.794+0000 7f4c11f96640 1 --2- 172.21.15.71:0/3361051307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c0c4350 0x7f4c0c0c52b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.794+0000 7f4c11f96640 1 -- 172.21.15.71:0/3361051307 >> 172.21.15.71:0/3361051307 conn(0x7f4c0c0c96e0 msgr2=0x7f4c0c0c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.794+0000 7f4c11f96640 1 -- 172.21.15.71:0/3361051307 shutdown_connections 2024-04-16T15:40:47.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.794+0000 7f4c11f96640 1 -- 172.21.15.71:0/3361051307 wait complete. 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.795+0000 7f4c11f96640 1 Processor -- start 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.795+0000 7f4c11f96640 1 -- start start 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.795+0000 7f4c11f96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c0c4350 0x7f4c0c1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.795+0000 7f4c11f96640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c0c153570 0x7f4c0c148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.795+0000 7f4c11f96640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c159740 0x7f4c0c148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.795+0000 7f4c11f96640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0c8ff0 con 0x7f4c0c159740 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c11f96640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0c64b0 con 0x7f4c0c0c4350 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c11f96640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0c0c6630 con 0x7f4c0c153570 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c159740 0x7f4c0c148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c159740 0x7f4c0c148ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52714/0 (socket says 172.21.15.71:52714) 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 -- 172.21.15.71:0/1015666516 learned_addr learned my addr 172.21.15.71:0/1015666516 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0b7fe640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c0c4350 0x7f4c0c1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c03fff640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c0c153570 0x7f4c0c148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 -- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c0c153570 msgr2=0x7f4c0c148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c0c153570 0x7f4c0c148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 -- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c0c4350 msgr2=0x7f4c0c1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c0c4350 0x7f4c0c1509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 -- 172.21.15.71:0/1015666516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c1ebe60 con 0x7f4c0c159740 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c03fff640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c0c153570 0x7f4c0c148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c0bfff640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c159740 0x7f4c0c148ac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c7a0 tx=0x7f4c0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c08ff9640 1 -- 172.21.15.71:0/1015666516 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04018070 con 0x7f4c0c159740 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c08ff9640 1 -- 172.21.15.71:0/1015666516 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0400a640 con 0x7f4c0c159740 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c08ff9640 1 -- 172.21.15.71:0/1015666516 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400aab0 con 0x7f4c0c159740 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c1ec140 con 0x7f4c0c159740 2024-04-16T15:40:47.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.796+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c1ec700 con 0x7f4c0c159740 2024-04-16T15:40:47.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.798+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0c1ec900 con 0x7f4c0c159740 2024-04-16T15:40:47.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.798+0000 7f4c08ff9640 1 -- 172.21.15.71:0/1015666516 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c0401b070 con 0x7f4c0c159740 2024-04-16T15:40:47.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.798+0000 7f4c08ff9640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4bec06c8f0 0x7f4bec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:47.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.798+0000 7f4c08ff9640 1 -- 172.21.15.71:0/1015666516 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4c0408b380 con 0x7f4c0c159740 2024-04-16T15:40:47.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.798+0000 7f4c0b7fe640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4bec06c8f0 0x7f4bec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:47.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.799+0000 7f4c0b7fe640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4bec06c8f0 0x7f4bec06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc004f60 tx=0x7f4bfc007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:47.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.804+0000 7f4c08ff9640 1 -- 172.21.15.71:0/1015666516 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c0c1ec900 con 0x7f4c0c159740 2024-04-16T15:40:47.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.980+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4c0c0c7c50 con 0x7f4c0c159740 2024-04-16T15:40:47.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.980+0000 7f4c08ff9640 1 -- 172.21.15.71:0/1015666516 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f4c04058eb0 con 0x7f4c0c159740 2024-04-16T15:40:47.980 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:47.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4bec06c8f0 msgr2=0x7f4bec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4bec06c8f0 0x7f4bec06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc004f60 tx=0x7f4bfc007e30 comp rx=0 tx=0).stop 2024-04-16T15:40:47.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c159740 msgr2=0x7f4c0c148ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:47.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c159740 0x7f4c0c148ac0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c7a0 tx=0x7f4c0400cc60 comp rx=0 tx=0).stop 2024-04-16T15:40:47.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 shutdown_connections 2024-04-16T15:40:47.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4bec06c8f0 0x7f4bec06edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c0c159740 0x7f4c0c148ac0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c0c153570 0x7f4c0c148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 --2- 172.21.15.71:0/1015666516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c0c4350 0x7f4c0c1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:47.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 >> 172.21.15.71:0/1015666516 conn(0x7f4c0c0c96e0 msgr2=0x7f4c0c0ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:47.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 shutdown_connections 2024-04-16T15:40:47.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:47.982+0000 7f4c11f96640 1 -- 172.21.15.71:0/1015666516 wait complete. 2024-04-16T15:40:47.993 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:40:48.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.100+0000 7f95d6688640 1 Processor -- start 2024-04-16T15:40:48.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.100+0000 7f95d6688640 1 -- start start 2024-04-16T15:40:48.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95d6688640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d014f840 0x7f95d00ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95d6688640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d00c3700 con 0x7f95d0152340 2024-04-16T15:40:48.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95d6688640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d00c2cb0 con 0x7f95d00acc00 2024-04-16T15:40:48.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95d6688640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f95d00c2780 con 0x7f95d014f840 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95d4c25640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f95d0152340 0x7f95d01527d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34790/0 (socket says 172.21.15.71:34790) 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95d4c25640 1 -- 172.21.15.71:0/1824048295 learned_addr learned my addr 172.21.15.71:0/1824048295 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238189227 0 0) 0x7f95d00c3700 con 0x7f95d0152340 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95b8004630 con 0x7f95d0152340 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95cffff640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d014f840 0x7f95d00ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3430420431 0 0) 0x7f95b8004630 con 0x7f95d0152340 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d014f840 msgr2=0x7f95d00ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95ceffd640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d014f840 0x7f95d00ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.101+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 >> v1:172.21.15.122:6789/0 conn(0x7f95d00acc00 legacy=0x7f95d00719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d00c39a0 con 0x7f95d0152340 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f95c80019f0 con 0x7f95d0152340 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 >> v1:172.21.15.71:6789/0 conn(0x7f95d0152340 legacy=0x7f95d01527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8004af0 0x7f95b8004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d014f840 0x7f95b8005910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8005e50 0x7f95b80062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95d00c2780 con 0x7f95b8005e50 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95b800cb80 con 0x7f95b8004af0 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f95b800cd80 con 0x7f95d014f840 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95d4c25640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8005e50 0x7f95b80062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95cffff640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8004af0 0x7f95b8004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95cf7fe640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d014f840 0x7f95b8005910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95d4c25640 1 -- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d014f840 msgr2=0x7f95b8005910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95d4c25640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d014f840 0x7f95b8005910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95d4c25640 1 -- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8004af0 msgr2=0x7f95b8004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95d4c25640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8004af0 0x7f95b8004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.102+0000 7f95d4c25640 1 -- 172.21.15.71:0/1824048295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b8004630 con 0x7f95b8005e50 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95d4c25640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8005e50 0x7f95b80062d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f95c800a880 tx=0x7f95b8006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c28405e9cb4dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c800a4a0 con 0x7f95b8005e50 2024-04-16T15:40:48.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c8034650 con 0x7f95b8005e50 2024-04-16T15:40:48.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95ceffd640 1 -- 172.21.15.71:0/1824048295 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8034a40 con 0x7f95b8005e50 2024-04-16T15:40:48.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95d6688640 1 -- 172.21.15.71:0/1824048295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8005e50 msgr2=0x7f95b80062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95d6688640 1 --2- 172.21.15.71:0/1824048295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b8005e50 0x7f95b80062d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f95c800a880 tx=0x7f95b8006940 comp rx=0 tx=0).stop 2024-04-16T15:40:48.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95d4c25640 1 -- 172.21.15.71:0/1824048295 reap_dead start 2024-04-16T15:40:48.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95d6688640 1 -- 172.21.15.71:0/1824048295 shutdown_connections 2024-04-16T15:40:48.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95d6688640 1 -- 172.21.15.71:0/1824048295 >> 172.21.15.71:0/1824048295 conn(0x7f95d00c7940 msgr2=0x7f95d00c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:48.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95d6688640 1 -- 172.21.15.71:0/1824048295 shutdown_connections 2024-04-16T15:40:48.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.103+0000 7f95d6688640 1 -- 172.21.15.71:0/1824048295 wait complete. 2024-04-16T15:40:48.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.105+0000 7f95d6688640 1 Processor -- start 2024-04-16T15:40:48.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.105+0000 7f95d6688640 1 -- start start 2024-04-16T15:40:48.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.105+0000 7f95d6688640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d00c09b0 0x7f95d0146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.105+0000 7f95d6688640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d01db630 0x7f95d0146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.105+0000 7f95d6688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d00aba90 0x7f95d01ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.105+0000 7f95d6688640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95d00720c0 con 0x7f95d01db630 2024-04-16T15:40:48.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.105+0000 7f95d6688640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0152ee0 con 0x7f95d00aba90 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.105+0000 7f95d6688640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f95d0072240 con 0x7f95d00c09b0 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d01db630 0x7f95d0146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d01db630 0x7f95d0146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52742/0 (socket says 172.21.15.71:52742) 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 -- 172.21.15.71:0/234385138 learned_addr learned my addr 172.21.15.71:0/234385138 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95d4c25640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d00aba90 0x7f95d01ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cffff640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d00c09b0 0x7f95d0146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 -- 172.21.15.71:0/234385138 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d00c09b0 msgr2=0x7f95d0146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d00c09b0 0x7f95d0146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 -- 172.21.15.71:0/234385138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d00aba90 msgr2=0x7f95d01ebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d00aba90 0x7f95d01ebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 -- 172.21.15.71:0/234385138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d01542e0 con 0x7f95d01db630 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95cf7fe640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d01db630 0x7f95d0146e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f95c400f7c0 tx=0x7f95c400fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95ccff9640 1 -- 172.21.15.71:0/234385138 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c40052e0 con 0x7f95d01db630 2024-04-16T15:40:48.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95ccff9640 1 -- 172.21.15.71:0/234385138 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c4018070 con 0x7f95d01db630 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95ccff9640 1 -- 172.21.15.71:0/234385138 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c400d970 con 0x7f95d01db630 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d00ac1d0 con 0x7f95d01db630 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.106+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d00ac490 con 0x7f95d01db630 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.107+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d0068be0 con 0x7f95d01db630 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.108+0000 7f95ccff9640 1 -- 172.21.15.71:0/234385138 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95c400db10 con 0x7f95d01db630 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.108+0000 7f95ccff9640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95bc06c8f0 0x7f95bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.108+0000 7f95cffff640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95bc06c8f0 0x7f95bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.108+0000 7f95ccff9640 1 -- 172.21.15.71:0/234385138 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f95c408bd90 con 0x7f95d01db630 2024-04-16T15:40:48.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.109+0000 7f95cffff640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95bc06c8f0 0x7f95bc06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009b00 tx=0x7f95c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.113+0000 7f95ccff9640 1 -- 172.21.15.71:0/234385138 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95c401a310 con 0x7f95d01db630 2024-04-16T15:40:48.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.291+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f95d00c3230 con 0x7f95d01db630 2024-04-16T15:40:48.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.292+0000 7f95ccff9640 1 -- 172.21.15.71:0/234385138 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f95c4059f10 con 0x7f95d01db630 2024-04-16T15:40:48.291 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95bc06c8f0 msgr2=0x7f95bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95bc06c8f0 0x7f95bc06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009b00 tx=0x7f95c0008040 comp rx=0 tx=0).stop 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d01db630 msgr2=0x7f95d0146e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d01db630 0x7f95d0146e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f95c400f7c0 tx=0x7f95c400fc80 comp rx=0 tx=0).stop 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 shutdown_connections 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95bc06c8f0 0x7f95bc06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95d00aba90 0x7f95d01ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d01db630 0x7f95d0146e60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 --2- 172.21.15.71:0/234385138 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95d00c09b0 0x7f95d0146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 >> 172.21.15.71:0/234385138 conn(0x7f95d00c7940 msgr2=0x7f95d0148de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 shutdown_connections 2024-04-16T15:40:48.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.294+0000 7f95d6688640 1 -- 172.21.15.71:0/234385138 wait complete. 2024-04-16T15:40:48.306 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:40:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.411+0000 7f678f4ba640 1 Processor -- start 2024-04-16T15:40:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.412+0000 7f678f4ba640 1 -- start start 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.412+0000 7f678f4ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f678806fac0 0x7f67880c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.412+0000 7f678f4ba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f67880ab290 0x7f678815d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.412+0000 7f678f4ba640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67880c27c0 con 0x7f6788155dc0 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.412+0000 7f678f4ba640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6788073f20 con 0x7f67880ab290 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.412+0000 7f678f4ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6788156ab0 con 0x7f678806fac0 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678da57640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f67880ab290 0x7f678815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678ca55640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f678806fac0 0x7f67880c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678d256640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f6788155dc0 0x7f67881561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:46516/0 (socket says 172.21.15.71:46516) 2024-04-16T15:40:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678d256640 1 -- 172.21.15.71:0/1153408493 learned_addr learned my addr 172.21.15.71:0/1153408493 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f6787fff640 1 -- 172.21.15.71:0/1153408493 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527430036 0 0) 0x7f67880c27c0 con 0x7f6788155dc0 2024-04-16T15:40:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f6787fff640 1 -- 172.21.15.71:0/1153408493 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6770003520 con 0x7f6788155dc0 2024-04-16T15:40:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678ca55640 1 -- 172.21.15.71:0/1153408493 >> v2:172.21.15.122:3300/0 conn(0x7f67880ab290 msgr2=0x7f678815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678ca55640 1 --2- 172.21.15.71:0/1153408493 >> v2:172.21.15.122:3300/0 conn(0x7f67880ab290 0x7f678815d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678ca55640 1 -- 172.21.15.71:0/1153408493 >> v1:172.21.15.122:6789/0 conn(0x7f6788155dc0 legacy=0x7f67881561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678ca55640 1 -- 172.21.15.71:0/1153408493 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67880c2aa0 con 0x7f678806fac0 2024-04-16T15:40:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.413+0000 7f678da57640 1 --2- 172.21.15.71:0/1153408493 >> v2:172.21.15.122:3300/0 conn(0x7f67880ab290 0x7f678815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.414+0000 7f678ca55640 1 --2- 172.21.15.71:0/1153408493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f678806fac0 0x7f67880c32a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f677c0097a0 tx=0x7f677c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4775323820947c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.414+0000 7f6787fff640 1 -- 172.21.15.71:0/1153408493 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c035020 con 0x7f678806fac0 2024-04-16T15:40:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.414+0000 7f6787fff640 1 -- 172.21.15.71:0/1153408493 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c031830 con 0x7f678806fac0 2024-04-16T15:40:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.414+0000 7f6787fff640 1 -- 172.21.15.71:0/1153408493 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c031be0 con 0x7f678806fac0 2024-04-16T15:40:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.415+0000 7f678f4ba640 1 -- 172.21.15.71:0/1153408493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f678806fac0 msgr2=0x7f67880c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.415+0000 7f678f4ba640 1 --2- 172.21.15.71:0/1153408493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f678806fac0 0x7f67880c32a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f677c0097a0 tx=0x7f677c02b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.415+0000 7f678f4ba640 1 -- 172.21.15.71:0/1153408493 shutdown_connections 2024-04-16T15:40:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.415+0000 7f678f4ba640 1 --2- 172.21.15.71:0/1153408493 >> v2:172.21.15.122:3300/0 conn(0x7f67880ab290 0x7f678815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.415+0000 7f678f4ba640 1 --2- 172.21.15.71:0/1153408493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f678806fac0 0x7f67880c32a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.415+0000 7f678f4ba640 1 -- 172.21.15.71:0/1153408493 >> 172.21.15.71:0/1153408493 conn(0x7f67880c3e10 msgr2=0x7f67880c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.415+0000 7f678f4ba640 1 -- 172.21.15.71:0/1153408493 shutdown_connections 2024-04-16T15:40:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.415+0000 7f678f4ba640 1 -- 172.21.15.71:0/1153408493 wait complete. 2024-04-16T15:40:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.416+0000 7f678f4ba640 1 Processor -- start 2024-04-16T15:40:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.416+0000 7f678f4ba640 1 -- start start 2024-04-16T15:40:48.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.416+0000 7f678f4ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67880ab290 0x7f67880c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.416+0000 7f678f4ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6788155dc0 0x7f67881dec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.416+0000 7f678f4ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67881ed140 0x7f67881df170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.416+0000 7f678f4ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f678806bd70 con 0x7f67880ab290 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.416+0000 7f678f4ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67880c18b0 con 0x7f6788155dc0 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.416+0000 7f678f4ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f678806bef0 con 0x7f67881ed140 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67880ab290 0x7f67880c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67880ab290 0x7f67880c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52754/0 (socket says 172.21.15.71:52754) 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 -- 172.21.15.71:0/3215575950 learned_addr learned my addr 172.21.15.71:0/3215575950 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678da57640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67881ed140 0x7f67881df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678ca55640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6788155dc0 0x7f67881dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 -- 172.21.15.71:0/3215575950 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67881ed140 msgr2=0x7f67881df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67881ed140 0x7f67881df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 -- 172.21.15.71:0/3215575950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6788155dc0 msgr2=0x7f67881dec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6788155dc0 0x7f67881dec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 -- 172.21.15.71:0/3215575950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67881df6b0 con 0x7f67880ab290 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678d256640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67880ab290 0x7f67880c5eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f677800caa0 tx=0x7f677800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f6785ffb640 1 -- 172.21.15.71:0/3215575950 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778017070 con 0x7f67880ab290 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f6785ffb640 1 -- 172.21.15.71:0/3215575950 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677800aba0 con 0x7f67880ab290 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f6785ffb640 1 -- 172.21.15.71:0/3215575950 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778012400 con 0x7f67880ab290 2024-04-16T15:40:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.417+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881e9f20 con 0x7f67880ab290 2024-04-16T15:40:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.418+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881ea4e0 con 0x7f67880ab290 2024-04-16T15:40:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.418+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6788068be0 con 0x7f67880ab290 2024-04-16T15:40:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.419+0000 7f6785ffb640 1 -- 172.21.15.71:0/3215575950 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f677801a070 con 0x7f67880ab290 2024-04-16T15:40:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.419+0000 7f6785ffb640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f677406c920 0x7f677406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.419+0000 7f678ca55640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f677406c920 0x7f677406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.419+0000 7f6785ffb640 1 -- 172.21.15.71:0/3215575950 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f677808bb30 con 0x7f67880ab290 2024-04-16T15:40:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.420+0000 7f678ca55640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f677406c920 0x7f677406ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f677c036000 tx=0x7f677c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.424+0000 7f6785ffb640 1 -- 172.21.15.71:0/3215575950 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6778059cb0 con 0x7f67880ab290 2024-04-16T15:40:48.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.602+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f67880ad490 con 0x7f67880ab290 2024-04-16T15:40:48.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.602+0000 7f6785ffb640 1 -- 172.21.15.71:0/3215575950 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f67780595b0 con 0x7f67880ab290 2024-04-16T15:40:48.602 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:48.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f677406c920 msgr2=0x7f677406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f677406c920 0x7f677406ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f677c036000 tx=0x7f677c008030 comp rx=0 tx=0).stop 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67880ab290 msgr2=0x7f67880c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67880ab290 0x7f67880c5eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f677800caa0 tx=0x7f677800cf60 comp rx=0 tx=0).stop 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 shutdown_connections 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f677406c920 0x7f677406ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67881ed140 0x7f67881df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6788155dc0 0x7f67881dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 --2- 172.21.15.71:0/3215575950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67880ab290 0x7f67880c5eb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.604+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 >> 172.21.15.71:0/3215575950 conn(0x7f67880c3e10 msgr2=0x7f67880abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.605+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 shutdown_connections 2024-04-16T15:40:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.605+0000 7f678f4ba640 1 -- 172.21.15.71:0/3215575950 wait complete. 2024-04-16T15:40:48.615 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:40:48.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.719+0000 7f4add3fa640 1 Processor -- start 2024-04-16T15:40:48.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.719+0000 7f4add3fa640 1 -- start start 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.719+0000 7f4add3fa640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad8159520 0x7f4ad8159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.719+0000 7f4add3fa640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4ad80c78d0 0x7f4ad806df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4add3fa640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad80ae4b0 con 0x7f4ad806e470 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4add3fa640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad80c9540 con 0x7f4ad80c78d0 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4add3fa640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad80c6940 con 0x7f4ad8159520 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ace7fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4ad80c78d0 0x7f4ad806df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ad6ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad8159520 0x7f4ad8159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ad77fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f4ad806e470 0x7f4ad80ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:46524/0 (socket says 172.21.15.71:46524) 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ace7fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4ad80c78d0 0x7f4ad806df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:37324/0 (socket says 172.21.15.71:37324) 2024-04-16T15:40:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ad77fe640 1 -- 172.21.15.71:0/2748519318 learned_addr learned my addr 172.21.15.71:0/2748519318 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ace7fc640 1 -- 172.21.15.71:0/2748519318 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad8159520 msgr2=0x7f4ad8159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ace7fc640 1 --2- 172.21.15.71:0/2748519318 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad8159520 0x7f4ad8159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ace7fc640 1 -- 172.21.15.71:0/2748519318 >> v1:172.21.15.122:6789/0 conn(0x7f4ad806e470 legacy=0x7f4ad80ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ace7fc640 1 -- 172.21.15.71:0/2748519318 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad815a030 con 0x7f4ad80c78d0 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ad67fc640 1 -- 172.21.15.71:0/2748519318 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685597810 0 0) 0x7f4ad80ae4b0 con 0x7f4ad806e470 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.720+0000 7f4ad6ffd640 1 --2- 172.21.15.71:0/2748519318 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad8159520 0x7f4ad8159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.721+0000 7f4ace7fc640 1 --2- 172.21.15.71:0/2748519318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad80c78d0 0x7f4ad806df30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80c96c0 tx=0x7f4ac002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3647117b0e7db987 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.721+0000 7f4ad67fc640 1 -- 172.21.15.71:0/2748519318 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0039070 con 0x7f4ad80c78d0 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.721+0000 7f4ad67fc640 1 -- 172.21.15.71:0/2748519318 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac00316f0 con 0x7f4ad80c78d0 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.721+0000 7f4ad67fc640 1 -- 172.21.15.71:0/2748519318 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0031ac0 con 0x7f4ad80c78d0 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.722+0000 7f4add3fa640 1 -- 172.21.15.71:0/2748519318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad80c78d0 msgr2=0x7f4ad806df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.722+0000 7f4add3fa640 1 --2- 172.21.15.71:0/2748519318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad80c78d0 0x7f4ad806df30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80c96c0 tx=0x7f4ac002b790 comp rx=0 tx=0).stop 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.722+0000 7f4add3fa640 1 -- 172.21.15.71:0/2748519318 shutdown_connections 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.722+0000 7f4add3fa640 1 --2- 172.21.15.71:0/2748519318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad80c78d0 0x7f4ad806df30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.722+0000 7f4add3fa640 1 --2- 172.21.15.71:0/2748519318 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad8159520 0x7f4ad8159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.722+0000 7f4add3fa640 1 -- 172.21.15.71:0/2748519318 >> 172.21.15.71:0/2748519318 conn(0x7f4ad80cb820 msgr2=0x7f4ad80cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:48.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.722+0000 7f4add3fa640 1 -- 172.21.15.71:0/2748519318 shutdown_connections 2024-04-16T15:40:48.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.722+0000 7f4add3fa640 1 -- 172.21.15.71:0/2748519318 wait complete. 2024-04-16T15:40:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.723+0000 7f4add3fa640 1 Processor -- start 2024-04-16T15:40:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.723+0000 7f4add3fa640 1 -- start start 2024-04-16T15:40:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4add3fa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ad806e470 0x7f4ad814e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4add3fa640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad80c78d0 0x7f4ad80ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4add3fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad8159520 0x7f4ad8151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4add3fa640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad80c9d50 con 0x7f4ad806e470 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4add3fa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad80c9a70 con 0x7f4ad8159520 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4add3fa640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad80b0cf0 con 0x7f4ad80c78d0 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ad806e470 0x7f4ad814e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ad806e470 0x7f4ad814e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52764/0 (socket says 172.21.15.71:52764) 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 -- 172.21.15.71:0/3847905038 learned_addr learned my addr 172.21.15.71:0/3847905038 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad77fe640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad8159520 0x7f4ad8151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ace7fc640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad80c78d0 0x7f4ad80ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 -- 172.21.15.71:0/3847905038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad80c78d0 msgr2=0x7f4ad80ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad80c78d0 0x7f4ad80ab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 -- 172.21.15.71:0/3847905038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad8159520 msgr2=0x7f4ad8151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad8159520 0x7f4ad8151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 -- 172.21.15.71:0/3847905038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad81e1e30 con 0x7f4ad806e470 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.724+0000 7f4ad6ffd640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ad806e470 0x7f4ad814e810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800e990 tx=0x7f4ac800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.725+0000 7f4acffff640 1 -- 172.21.15.71:0/3847905038 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8017070 con 0x7f4ad806e470 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.725+0000 7f4acffff640 1 -- 172.21.15.71:0/3847905038 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac800c9e0 con 0x7f4ad806e470 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.725+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad81e2110 con 0x7f4ad806e470 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.725+0000 7f4acffff640 1 -- 172.21.15.71:0/3847905038 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac800cdb0 con 0x7f4ad806e470 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.725+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad8151fb0 con 0x7f4ad806e470 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.726+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad8074060 con 0x7f4ad806e470 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.726+0000 7f4acffff640 1 -- 172.21.15.71:0/3847905038 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ac801a070 con 0x7f4ad806e470 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.726+0000 7f4acffff640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4abc06c8f0 0x7f4abc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.726+0000 7f4ace7fc640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4abc06c8f0 0x7f4abc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.726+0000 7f4acffff640 1 -- 172.21.15.71:0/3847905038 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4ac808c000 con 0x7f4ad806e470 2024-04-16T15:40:48.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.727+0000 7f4ace7fc640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4abc06c8f0 0x7f4abc06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0036000 tx=0x7f4ac0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:48.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.731+0000 7f4acffff640 1 -- 172.21.15.71:0/3847905038 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ac8059ab0 con 0x7f4ad806e470 2024-04-16T15:40:48.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.945+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ad806df30 con 0x7f4ad806e470 2024-04-16T15:40:48.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.945+0000 7f4acffff640 1 -- 172.21.15.71:0/3847905038 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f4ad806df30 con 0x7f4ad806e470 2024-04-16T15:40:48.945 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:48.945 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:40.120873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:40:48.946 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-04-16T15:40:48.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4abc06c8f0 msgr2=0x7f4abc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4abc06c8f0 0x7f4abc06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0036000 tx=0x7f4ac0008030 comp rx=0 tx=0).stop 2024-04-16T15:40:48.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ad806e470 msgr2=0x7f4ad814e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:48.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ad806e470 0x7f4ad814e810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800e990 tx=0x7f4ac800ee50 comp rx=0 tx=0).stop 2024-04-16T15:40:48.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 shutdown_connections 2024-04-16T15:40:48.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4abc06c8f0 0x7f4abc06edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad8159520 0x7f4ad8151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ad80c78d0 0x7f4ad80ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 --2- 172.21.15.71:0/3847905038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ad806e470 0x7f4ad814e810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:48.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 >> 172.21.15.71:0/3847905038 conn(0x7f4ad80cb820 msgr2=0x7f4ad80cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:48.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 shutdown_connections 2024-04-16T15:40:48.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:48.947+0000 7f4add3fa640 1 -- 172.21.15.71:0/3847905038 wait complete. 2024-04-16T15:40:48.959 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:49.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.061+0000 7fc1e5bbd640 1 Processor -- start 2024-04-16T15:40:49.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.061+0000 7fc1e5bbd640 1 -- start start 2024-04-16T15:40:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1e5bbd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc1e00c4350 0x7fc1e00c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1e5bbd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0159740 0x7fc1e0159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1e5bbd640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e00c82d0 con 0x7fc1e0153570 2024-04-16T15:40:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1e5bbd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e00c41a0 con 0x7fc1e00c4350 2024-04-16T15:40:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1e5bbd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e0155d70 con 0x7fc1e0159740 2024-04-16T15:40:49.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1df7fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fc1e0153570 0x7fc1e00c5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34796/0 (socket says 172.21.15.71:34796) 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1df7fe640 1 -- 172.21.15.71:0/3593464450 learned_addr learned my addr 172.21.15.71:0/3593464450 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1deffd640 1 --2- 172.21.15.71:0/3593464450 >> v2:172.21.15.122:3300/0 conn(0x7fc1e00c4350 0x7fc1e00c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674577328 0 0) 0x7fc1e00c82d0 con 0x7fc1e0153570 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1c8003520 con 0x7fc1e0153570 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.062+0000 7fc1dffff640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0159740 0x7fc1e0159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1495675012 0 0) 0x7fc1c8003520 con 0x7fc1e0153570 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0159740 msgr2=0x7fc1e0159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0159740 0x7fc1e0159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 >> v2:172.21.15.122:3300/0 conn(0x7fc1e00c4350 msgr2=0x7fc1e00c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 --2- 172.21.15.71:0/3593464450 >> v2:172.21.15.122:3300/0 conn(0x7fc1e00c4350 0x7fc1e00c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e00c8d50 con 0x7fc1e0153570 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fc1d00019f0 con 0x7fc1e0153570 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 >> v1:172.21.15.71:6789/0 conn(0x7fc1e0153570 legacy=0x7fc1e00c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0159740 0x7fc1c80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1c8003f30 0x7fc1c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8004d40 0x7fc1c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e0155d70 con 0x7fc1c8003f30 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c800ba20 con 0x7fc1c8004d40 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c800bc20 con 0x7fc1e0159740 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1deffd640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1c8003f30 0x7fc1c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1df7fe640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0159740 0x7fc1c80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1dffff640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8004d40 0x7fc1c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1deffd640 1 -- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0159740 msgr2=0x7fc1c80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1deffd640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0159740 0x7fc1c80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1deffd640 1 -- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8004d40 msgr2=0x7fc1c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1deffd640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8004d40 0x7fc1c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.063+0000 7fc1deffd640 1 -- 172.21.15.71:0/3593464450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c8003520 con 0x7fc1c8003f30 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.064+0000 7fc1deffd640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1c8003f30 0x7fc1c8004800 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009ac0 tx=0x7fc1c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf064c6010f24a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.064+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4039070 con 0x7fc1c8003f30 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.064+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d402c710 con 0x7fc1c8003f30 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.064+0000 7fc1de7fc640 1 -- 172.21.15.71:0/3593464450 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4009690 con 0x7fc1c8003f30 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.065+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/3593464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1c8003f30 msgr2=0x7fc1c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.065+0000 7fc1e5bbd640 1 --2- 172.21.15.71:0/3593464450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1c8003f30 0x7fc1c8004800 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009ac0 tx=0x7fc1c8005830 comp rx=0 tx=0).stop 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.065+0000 7fc1dffff640 1 -- 172.21.15.71:0/3593464450 reap_dead start 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.065+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/3593464450 shutdown_connections 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.065+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/3593464450 >> 172.21.15.71:0/3593464450 conn(0x7fc1e00c96e0 msgr2=0x7fc1e00c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.065+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/3593464450 shutdown_connections 2024-04-16T15:40:49.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.065+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/3593464450 wait complete. 2024-04-16T15:40:49.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.066+0000 7fc1e5bbd640 1 Processor -- start 2024-04-16T15:40:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.066+0000 7fc1e5bbd640 1 -- start start 2024-04-16T15:40:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1e5bbd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8003f30 0x7fc1e00a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1e5bbd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e014ee70 0x7fc1e014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1e5bbd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0071170 0x7fc1e00715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1e5bbd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e00c64b0 con 0x7fc1e014ee70 2024-04-16T15:40:49.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1e5bbd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e00c8ff0 con 0x7fc1c8003f30 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1e5bbd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e00c6630 con 0x7fc1e0071170 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e014ee70 0x7fc1e014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e014ee70 0x7fc1e014e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52780/0 (socket says 172.21.15.71:52780) 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 -- 172.21.15.71:0/1080618484 learned_addr learned my addr 172.21.15.71:0/1080618484 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1df7fe640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8003f30 0x7fc1e00a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1dffff640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0071170 0x7fc1e00715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 -- 172.21.15.71:0/1080618484 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0071170 msgr2=0x7fc1e00715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0071170 0x7fc1e00715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 -- 172.21.15.71:0/1080618484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8003f30 msgr2=0x7fc1e00a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8003f30 0x7fc1e00a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 -- 172.21.15.71:0/1080618484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e00a8ba0 con 0x7fc1e014ee70 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1deffd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e014ee70 0x7fc1e014e750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4000c00 tx=0x7fc1d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1c7fff640 1 -- 172.21.15.71:0/1080618484 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4039050 con 0x7fc1e014ee70 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1c7fff640 1 -- 172.21.15.71:0/1080618484 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d4045070 con 0x7fc1e014ee70 2024-04-16T15:40:49.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e006ce40 con 0x7fc1e014ee70 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1c7fff640 1 -- 172.21.15.71:0/1080618484 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d40044e0 con 0x7fc1e014ee70 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.067+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e01ebb00 con 0x7fc1e014ee70 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.068+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e0068d20 con 0x7fc1e014ee70 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.069+0000 7fc1c7fff640 1 -- 172.21.15.71:0/1080618484 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1d4003a50 con 0x7fc1e014ee70 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.069+0000 7fc1c7fff640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc1cc06c8f0 0x7fc1cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.069+0000 7fc1c7fff640 1 -- 172.21.15.71:0/1080618484 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc1d40ae080 con 0x7fc1e014ee70 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.069+0000 7fc1df7fe640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc1cc06c8f0 0x7fc1cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.070+0000 7fc1df7fe640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc1cc06c8f0 0x7fc1cc06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0002230 tx=0x7fc1d0006040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:49.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.075+0000 7fc1c7fff640 1 -- 172.21.15.71:0/1080618484 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1d407c200 con 0x7fc1e014ee70 2024-04-16T15:40:49.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.249+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1e00c7c50 con 0x7fc1e014ee70 2024-04-16T15:40:49.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.250+0000 7fc1c7fff640 1 -- 172.21.15.71:0/1080618484 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7fc1d407bb00 con 0x7fc1e014ee70 2024-04-16T15:40:49.250 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc1cc06c8f0 msgr2=0x7fc1cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc1cc06c8f0 0x7fc1cc06edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0002230 tx=0x7fc1d0006040 comp rx=0 tx=0).stop 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e014ee70 msgr2=0x7fc1e014e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e014ee70 0x7fc1e014e750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4000c00 tx=0x7fc1d4004730 comp rx=0 tx=0).stop 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 shutdown_connections 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc1cc06c8f0 0x7fc1cc06edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc1e0071170 0x7fc1e00715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e014ee70 0x7fc1e014e750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 --2- 172.21.15.71:0/1080618484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1c8003f30 0x7fc1e00a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 >> 172.21.15.71:0/1080618484 conn(0x7fc1e00c96e0 msgr2=0x7fc1e00c4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:49.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 shutdown_connections 2024-04-16T15:40:49.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.252+0000 7fc1e5bbd640 1 -- 172.21.15.71:0/1080618484 wait complete. 2024-04-16T15:40:49.263 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:40:49.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.367+0000 7fa8ce674640 1 Processor -- start 2024-04-16T15:40:49.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.367+0000 7fa8ce674640 1 -- start start 2024-04-16T15:40:49.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8ce674640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c80c52b0 0x7fa8c8159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8ce674640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c80c82d0 con 0x7fa8c80c4350 2024-04-16T15:40:49.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8ce674640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c8155d70 con 0x7fa8c8153570 2024-04-16T15:40:49.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8ce674640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c80c41a0 con 0x7fa8c80c52b0 2024-04-16T15:40:49.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8c77fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fa8c80c4350 0x7fa8c8159740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34800/0 (socket says 172.21.15.71:34800) 2024-04-16T15:40:49.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8c77fe640 1 -- 172.21.15.71:0/2936677172 learned_addr learned my addr 172.21.15.71:0/2936677172 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:49.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8ccc11640 1 --2- 172.21.15.71:0/2936677172 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c80c52b0 0x7fa8c8159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8c6ffd640 1 -- 172.21.15.71:0/2936677172 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1401349420 0 0) 0x7fa8c80c82d0 con 0x7fa8c80c4350 2024-04-16T15:40:49.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8c6ffd640 1 -- 172.21.15.71:0/2936677172 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8b0003520 con 0x7fa8c80c4350 2024-04-16T15:40:49.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8c6ffd640 1 -- 172.21.15.71:0/2936677172 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3065777868 0 0) 0x7fa8c8155d70 con 0x7fa8c8153570 2024-04-16T15:40:49.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8ccc11640 1 -- 172.21.15.71:0/2936677172 >> v1:172.21.15.122:6789/0 conn(0x7fa8c8153570 legacy=0x7fa8c80c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8ccc11640 1 -- 172.21.15.71:0/2936677172 >> v1:172.21.15.71:6789/0 conn(0x7fa8c80c4350 legacy=0x7fa8c8159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.368+0000 7fa8ccc11640 1 -- 172.21.15.71:0/2936677172 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c80c8d50 con 0x7fa8c80c52b0 2024-04-16T15:40:49.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.369+0000 7fa8ccc11640 1 --2- 172.21.15.71:0/2936677172 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c80c52b0 0x7fa8c8159e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009dc0 tx=0x7fa8c002be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f09a5a6088e8788 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.369+0000 7fa8c6ffd640 1 -- 172.21.15.71:0/2936677172 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0039070 con 0x7fa8c80c52b0 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.369+0000 7fa8c6ffd640 1 -- 172.21.15.71:0/2936677172 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c002cbb0 con 0x7fa8c80c52b0 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.369+0000 7fa8c6ffd640 1 -- 172.21.15.71:0/2936677172 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0034440 con 0x7fa8c80c52b0 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.370+0000 7fa8ce674640 1 -- 172.21.15.71:0/2936677172 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c80c52b0 msgr2=0x7fa8c8159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.370+0000 7fa8ce674640 1 --2- 172.21.15.71:0/2936677172 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c80c52b0 0x7fa8c8159e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009dc0 tx=0x7fa8c002be00 comp rx=0 tx=0).stop 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.370+0000 7fa8ce674640 1 -- 172.21.15.71:0/2936677172 shutdown_connections 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.370+0000 7fa8ce674640 1 --2- 172.21.15.71:0/2936677172 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c80c52b0 0x7fa8c8159e50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.370+0000 7fa8ce674640 1 -- 172.21.15.71:0/2936677172 >> 172.21.15.71:0/2936677172 conn(0x7fa8c80c96e0 msgr2=0x7fa8c80c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.370+0000 7fa8ce674640 1 -- 172.21.15.71:0/2936677172 shutdown_connections 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.370+0000 7fa8ce674640 1 -- 172.21.15.71:0/2936677172 wait complete. 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.371+0000 7fa8ce674640 1 Processor -- start 2024-04-16T15:40:49.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.371+0000 7fa8ce674640 1 -- start start 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8ce674640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80c4350 0x7fa8c8071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8ce674640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c80c52b0 0x7fa8c806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8ce674640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c8153570 0x7fa8c806f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8ce674640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c80c64b0 con 0x7fa8c80c4350 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8ce674640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c80c8ff0 con 0x7fa8c80c52b0 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8ce674640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c80c6630 con 0x7fa8c8153570 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80c4350 0x7fa8c8071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80c4350 0x7fa8c8071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52792/0 (socket says 172.21.15.71:52792) 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 -- 172.21.15.71:0/3049857279 learned_addr learned my addr 172.21.15.71:0/3049857279 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c77fe640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c80c52b0 0x7fa8c806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8ccc11640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c8153570 0x7fa8c806f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 -- 172.21.15.71:0/3049857279 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c8153570 msgr2=0x7fa8c806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c8153570 0x7fa8c806f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 -- 172.21.15.71:0/3049857279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c80c52b0 msgr2=0x7fa8c806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c80c52b0 0x7fa8c806f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 -- 172.21.15.71:0/3049857279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c80abf20 con 0x7fa8c80c4350 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c7fff640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80c4350 0x7fa8c8071330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800c440 tx=0x7fa8b800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c4ff9640 1 -- 172.21.15.71:0/3049857279 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8017070 con 0x7fa8c80c4350 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.372+0000 7fa8c4ff9640 1 -- 172.21.15.71:0/3049857279 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b8005480 con 0x7fa8c80c4350 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.373+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c81ebb50 con 0x7fa8c80c4350 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.373+0000 7fa8c4ff9640 1 -- 172.21.15.71:0/3049857279 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8003830 con 0x7fa8c80c4350 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.373+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c81ec110 con 0x7fa8c80c4350 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.374+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c8158240 con 0x7fa8c80c4350 2024-04-16T15:40:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.374+0000 7fa8c4ff9640 1 -- 172.21.15.71:0/3049857279 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8b801a070 con 0x7fa8c80c4350 2024-04-16T15:40:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.375+0000 7fa8c4ff9640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b406c8f0 0x7fa8b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.375+0000 7fa8c77fe640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b406c8f0 0x7fa8b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.375+0000 7fa8c4ff9640 1 -- 172.21.15.71:0/3049857279 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa8b808b120 con 0x7fa8c80c4350 2024-04-16T15:40:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.375+0000 7fa8c77fe640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b406c8f0 0x7fa8b406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa8c81577a0 tx=0x7fa8bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:49.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.379+0000 7fa8c4ff9640 1 -- 172.21.15.71:0/3049857279 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8c8158240 con 0x7fa8c80c4350 2024-04-16T15:40:49.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.590+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8c80c7c50 con 0x7fa8c80c4350 2024-04-16T15:40:49.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.590+0000 7fa8c4ff9640 1 -- 172.21.15.71:0/3049857279 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fa8c80abf20 con 0x7fa8c80c4350 2024-04-16T15:40:49.590 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:49.590 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:40.120873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:40:49.591 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-04-16T15:40:49.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b406c8f0 msgr2=0x7fa8b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b406c8f0 0x7fa8b406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa8c81577a0 tx=0x7fa8bc009040 comp rx=0 tx=0).stop 2024-04-16T15:40:49.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80c4350 msgr2=0x7fa8c8071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:49.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80c4350 0x7fa8c8071330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800c440 tx=0x7fa8b800c900 comp rx=0 tx=0).stop 2024-04-16T15:40:49.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 shutdown_connections 2024-04-16T15:40:49.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b406c8f0 0x7fa8b406edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8c8153570 0x7fa8c806f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c80c52b0 0x7fa8c806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 --2- 172.21.15.71:0/3049857279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c80c4350 0x7fa8c8071330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:49.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 >> 172.21.15.71:0/3049857279 conn(0x7fa8c80c96e0 msgr2=0x7fa8c81577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:49.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 shutdown_connections 2024-04-16T15:40:49.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:49.592+0000 7fa8ce674640 1 -- 172.21.15.71:0/3049857279 wait complete. 2024-04-16T15:40:50.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-16T15:40:23.878007+0000', 'modified': '2024-04-16T15:40:40.120873+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6835/153100987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6833', 'nonce': 153100987}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 153100987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4296, 'qdb_cluster': [4296]} 2024-04-16T15:40:50.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-16T15:40:50.006 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T15:40:50.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.112+0000 7f9cc217c640 1 Processor -- start 2024-04-16T15:40:50.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.112+0000 7f9cc217c640 1 -- start start 2024-04-16T15:40:50.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cc217c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc06cbb0 0x7f9cbc0c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cc217c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9cbc0726c0 0x7f9cbc0c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cc217c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cbc0c5bf0 con 0x7f9cbc0c3570 2024-04-16T15:40:50.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cc217c640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f9cbc0c59a0 con 0x7f9cbc0726c0 2024-04-16T15:40:50.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cc217c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9cbc0c6800 con 0x7f9cbc06cbb0 2024-04-16T15:40:50.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cbaffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9cbc0726c0 0x7f9cbc0c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cbaffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f9cbc0726c0 0x7f9cbc0c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52808/0 (socket says 172.21.15.71:52808) 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cbaffd640 1 -- 172.21.15.71:0/3010395875 learned_addr learned my addr 172.21.15.71:0/3010395875 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.113+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/3010395875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc06cbb0 0x7f9cbc0c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.114+0000 7f9cbaffd640 1 -- 172.21.15.71:0/3010395875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc06cbb0 msgr2=0x7f9cbc0c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.114+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/3010395875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc06cbb0 0x7f9cbc0c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.114+0000 7f9cbaffd640 1 -- 172.21.15.71:0/3010395875 >> v1:172.21.15.122:6789/0 conn(0x7f9cbc0c3570 legacy=0x7f9cbc0c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.114+0000 7f9cbaffd640 1 -- 172.21.15.71:0/3010395875 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc0c5ed0 con 0x7f9cbc0726c0 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.114+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/3010395875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0726c0 0x7f9cbc0c3030 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009870 tx=0x7f9cb002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885b9333c6ce3ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.114+0000 7f9cba7fc640 1 -- 172.21.15.71:0/3010395875 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0035020 con 0x7f9cbc0726c0 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cba7fc640 1 -- 172.21.15.71:0/3010395875 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb0031640 con 0x7f9cbc0726c0 2024-04-16T15:40:50.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cba7fc640 1 -- 172.21.15.71:0/3010395875 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb002d040 con 0x7f9cbc0726c0 2024-04-16T15:40:50.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cc217c640 1 -- 172.21.15.71:0/3010395875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0726c0 msgr2=0x7f9cbc0c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3010395875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0726c0 0x7f9cbc0c3030 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009870 tx=0x7f9cb002b8f0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cc217c640 1 -- 172.21.15.71:0/3010395875 shutdown_connections 2024-04-16T15:40:50.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3010395875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0726c0 0x7f9cbc0c3030 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3010395875 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc06cbb0 0x7f9cbc0c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cc217c640 1 -- 172.21.15.71:0/3010395875 >> 172.21.15.71:0/3010395875 conn(0x7f9cbc0c45a0 msgr2=0x7f9cbc0c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cc217c640 1 -- 172.21.15.71:0/3010395875 shutdown_connections 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.115+0000 7f9cc217c640 1 -- 172.21.15.71:0/3010395875 wait complete. 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.116+0000 7f9cc217c640 1 Processor -- start 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.116+0000 7f9cc217c640 1 -- start start 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cc217c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cbc06cbb0 0x7f9cbc1ef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cc217c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc0726c0 0x7f9cbc1dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cc217c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0c3570 0x7f9cbc1d6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cc217c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc06cf90 con 0x7f9cbc0c3570 2024-04-16T15:40:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cc217c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc0c6490 con 0x7f9cbc06cbb0 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cc217c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9cbc06d110 con 0x7f9cbc0726c0 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0c3570 0x7f9cbc1d6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0c3570 0x7f9cbc1d6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52820/0 (socket says 172.21.15.71:52820) 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cbbfff640 1 -- 172.21.15.71:0/3190085804 learned_addr learned my addr 172.21.15.71:0/3190085804 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cbc06cbb0 0x7f9cbc1ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.117+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc0726c0 0x7f9cbc1dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9cbbfff640 1 -- 172.21.15.71:0/3190085804 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc0726c0 msgr2=0x7f9cbc1dbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc0726c0 0x7f9cbc1dbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9cbbfff640 1 -- 172.21.15.71:0/3190085804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cbc06cbb0 msgr2=0x7f9cbc1ef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cbc06cbb0 0x7f9cbc1ef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9cbbfff640 1 -- 172.21.15.71:0/3190085804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc1ec470 con 0x7f9cbc0c3570 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0c3570 0x7f9cbc1d6c30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400c680 tx=0x7f9cb400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9ca3fff640 1 -- 172.21.15.71:0/3190085804 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4003b70 con 0x7f9cbc0c3570 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9ca3fff640 1 -- 172.21.15.71:0/3190085804 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb4003d10 con 0x7f9cbc0c3570 2024-04-16T15:40:50.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9cc217c640 1 -- 172.21.15.71:0/3190085804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc1ec710 con 0x7f9cbc0c3570 2024-04-16T15:40:50.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9ca3fff640 1 -- 172.21.15.71:0/3190085804 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4005310 con 0x7f9cbc0c3570 2024-04-16T15:40:50.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.118+0000 7f9cc217c640 1 -- 172.21.15.71:0/3190085804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc1dc180 con 0x7f9cbc0c3570 2024-04-16T15:40:50.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.119+0000 7f9ca3fff640 1 -- 172.21.15.71:0/3190085804 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cb40054b0 con 0x7f9cbc0c3570 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.120+0000 7f9ca3fff640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9ca806c8f0 0x7f9ca806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.120+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9ca806c8f0 0x7f9ca806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.120+0000 7f9ca3fff640 1 -- 172.21.15.71:0/3190085804 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9cb408bbb0 con 0x7f9cbc0c3570 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.120+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9ca806c8f0 0x7f9ca806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc1e9790 tx=0x7f9cac009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.121+0000 7f9cc49fd740 1 Processor -- start 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.121+0000 7f9cc49fd740 1 -- start start 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f590 0x5623060b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562305fe39c0 con 0x562306080620 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562305f8a650 con 0x56230607f590 2024-04-16T15:40:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x562305fdf980 con 0x56230607f970 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060bb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52834/0 (socket says 172.21.15.71:52834) 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 -- 172.21.15.71:0/2255397 learned_addr learned my addr 172.21.15.71:0/2255397 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f590 0x5623060b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 -- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 msgr2=0x5623060ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 -- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f590 msgr2=0x5623060b9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f590 0x5623060b9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 -- 172.21.15.71:0/2255397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562305cf62d0 con 0x562306080620 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f590 0x5623060b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060bb3f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4059070 tx=0x7f9cb4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cda2d091c1d50e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.122+0000 7f9ca17fa640 1 -- 172.21.15.71:0/2255397 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4088070 con 0x562306080620 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9ca17fa640 1 -- 172.21.15.71:0/2255397 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb40576e0 con 0x562306080620 2024-04-16T15:40:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9ca17fa640 1 -- 172.21.15.71:0/2255397 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4055cb0 con 0x562306080620 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2255397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 msgr2=0x5623060bb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060bb3f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4059070 tx=0x7f9cb4011000 comp rx=0 tx=0).stop 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2255397 shutdown_connections 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060bb3f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2255397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f590 0x5623060b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2255397 >> 172.21.15.71:0/2255397 conn(0x56230607bc60 msgr2=0x5623060a0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2255397 shutdown_connections 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.123+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2255397 wait complete. 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.124+0000 7f9cc49fd740 1 -- start start 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.124+0000 7f9cc49fd740 1 Processor -- start 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56230607f970 0x5623060a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x562306080620 0x5623060cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x5623060a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5623060be1b0 con 0x56230607f970 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562305f9da10 con 0x5623060a29c0 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x5623060bcd90 con 0x562306080620 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 10 client.0 _ll_register_callbacks cb 0x562305f9d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56230607f970 0x5623060a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56230607f970 0x5623060a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52844/0 (socket says 172.21.15.71:52844) 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 -- 172.21.15.71:0/2526331581 learned_addr learned my addr 172.21.15.71:0/2526331581 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x562306080620 0x5623060cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x5623060a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 -- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x562306080620 msgr2=0x5623060cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x562306080620 0x5623060cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 -- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 msgr2=0x5623060a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x5623060a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 -- 172.21.15.71:0/2526331581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562305dc06b0 con 0x56230607f970 2024-04-16T15:40:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x562306080620 0x5623060cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x5623060a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56230607f970 0x5623060a3dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9cac011fd0 tx=0x7f9cac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a42aa95b4cdb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9c9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2526331581 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac027020 con 0x56230607f970 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2526331581 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9cac023450 con 0x56230607f970 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2526331581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb40576e0 con 0x56230607f970 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2526331581 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f9cac031050 con 0x56230607f970 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 10 client.4800 fetch_fsmap learned FSMap version 9 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.125+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2526331581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5623060a19f0 con 0x56230607f970 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2526331581 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb40576e0 con 0x56230607f970 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2526331581 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f9cac034020 con 0x56230607f970 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9cc49fd740 10 client.4800 fetch_fsmap finished waiting for FSMap version 9 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9cc49fd740 10 client.4800 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9cc49fd740 20 client.4800 populate_metadata read hostname 'smithi071' 2024-04-16T15:40:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x5623060cc6f0 0x56230609db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9cc49fd740 4 client.4800 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2526331581 --> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562305ff13e0 con 0x5623060cc6f0 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x5623060cc6f0 0x56230609db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x5623060cc6f0 0x56230609db60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180d2e87329f23fe server_cookie=64a44508a6d0fd6f in_seq=0 out_seq=0 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.126+0000 7f9c9a7fc640 10 client.4800 ms_handle_connect on v2:172.21.15.122:6833/153100987 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2526331581 <== mds.0 v2:172.21.15.122:6833/153100987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562305ff13e0 con 0x5623060cc6f0 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9c9a7fc640 10 client.4800 handle_command_reply: tid=0 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 client.4800 shutdown 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2526331581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56230607f970 msgr2=0x5623060a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56230607f970 0x5623060a3dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9cac011fd0 tx=0x7f9cac0046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2526331581 shutdown_connections 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x5623060a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x5623060cc6f0 0x56230609db60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x562306080620 0x5623060cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2526331581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56230607f970 0x5623060a3dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9c9a7fc640 0 client.4800 ms_handle_reset on v2:172.21.15.122:6833/153100987 2024-04-16T15:40:50.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2526331581 >> 172.21.15.71:0/2526331581 conn(0x56230607f590 msgr2=0x5623060bcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:50.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2526331581 shutdown_connections 2024-04-16T15:40:50.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2526331581 wait complete. 2024-04-16T15:40:50.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.127+0000 7f9cc49fd740 20 client.4800 trim_cache size 0 max 16384 2024-04-16T15:40:50.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.145+0000 7f9cc49fd740 1 Processor -- start 2024-04-16T15:40:50.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.145+0000 7f9cc49fd740 1 -- start start 2024-04-16T15:40:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f970 0x5623060bf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060a0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5623060a29c0 0x5623060797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc1f8920 con 0x562306080620 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc1f7db0 con 0x56230607f970 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9cbc1f82d0 con 0x5623060a29c0 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cbaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cbaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52852/0 (socket says 172.21.15.71:52852) 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cbaffd640 1 -- 172.21.15.71:0/1978550011 learned_addr learned my addr 172.21.15.71:0/1978550011 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cbaffd640 1 -- 172.21.15.71:0/1978550011 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5623060a29c0 msgr2=0x5623060797f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/1978550011 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5623060a29c0 0x5623060797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cbaffd640 1 -- 172.21.15.71:0/1978550011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f970 msgr2=0x5623060bf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/1978550011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f970 0x5623060bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.146+0000 7f9cbaffd640 1 -- 172.21.15.71:0/1978550011 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562306075aa0 con 0x562306080620 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/1978550011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060a0620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00316e0 tx=0x7f9cb0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4c31d89f8a8f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9ca0ff9640 1 -- 172.21.15.71:0/1978550011 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb004e070 con 0x562306080620 2024-04-16T15:40:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9ca0ff9640 1 -- 172.21.15.71:0/1978550011 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb00040c0 con 0x562306080620 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9ca0ff9640 1 -- 172.21.15.71:0/1978550011 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0035040 con 0x562306080620 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 -- 172.21.15.71:0/1978550011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 msgr2=0x5623060a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/1978550011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060a0620 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00316e0 tx=0x7f9cb0008030 comp rx=0 tx=0).stop 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 -- 172.21.15.71:0/1978550011 shutdown_connections 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/1978550011 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5623060a29c0 0x5623060797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/1978550011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060a0620 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/1978550011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56230607f970 0x5623060bf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 -- 172.21.15.71:0/1978550011 >> 172.21.15.71:0/1978550011 conn(0x56230607f590 msgr2=0x5623060a3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 -- 172.21.15.71:0/1978550011 shutdown_connections 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.147+0000 7f9cc49fd740 1 -- 172.21.15.71:0/1978550011 wait complete. 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.148+0000 7f9cc49fd740 1 -- start start 2024-04-16T15:40:50.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.148+0000 7f9cc49fd740 1 Processor -- start 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x562306133220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5623060a4910 con 0x562306080620 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5623060a3420 con 0x5623060a29c0 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x5623060cba60 con 0x56230607f970 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 10 client.0 _ll_register_callbacks cb 0x7f9cac004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060d54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52858/0 (socket says 172.21.15.71:52858) 2024-04-16T15:40:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 -- 172.21.15.71:0/2066237958 learned_addr learned my addr 172.21.15.71:0/2066237958 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x562306133220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 -- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 msgr2=0x5623060b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 -- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 msgr2=0x562306133220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x562306133220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 -- 172.21.15.71:0/2066237958 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623060d5f50 con 0x562306080620 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbbfff640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x562306133220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cbaffd640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060d54f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0000c00 tx=0x7f9cb002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a39bebce16992f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9c9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2066237958 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0057060 con 0x562306080620 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2066237958 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9cb0031810 con 0x562306080620 2024-04-16T15:40:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2066237958 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb00040c0 con 0x562306080620 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2066237958 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f9cb005d050 con 0x562306080620 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 10 client.4806 fetch_fsmap learned FSMap version 9 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.149+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2066237958 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5623060b8590 con 0x562306080620 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2066237958 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb00040c0 con 0x562306080620 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2066237958 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f9cb0060020 con 0x562306080620 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cc49fd740 10 client.4806 fetch_fsmap finished waiting for FSMap version 9 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cc49fd740 10 client.4806 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cc49fd740 20 client.4806 populate_metadata read hostname 'smithi071' 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x562306139620 0x562306139a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cc49fd740 4 client.4806 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2066237958 --> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] -- command(tid 0: {"prefix": "status"}) -- 0x5623060b2a20 con 0x562306139620 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x562306139620 0x562306139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cbb7fe640 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x562306139620 0x562306139a00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa84a15748ea3ce6 server_cookie=f48a88d694151e7a in_seq=0 out_seq=0 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9c9a7fc640 10 client.4806 ms_handle_connect on v2:172.21.15.122:6833/153100987 2024-04-16T15:40:50.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9c9a7fc640 1 -- 172.21.15.71:0/2066237958 <== mds.0 v2:172.21.15.122:6833/153100987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x5623060b2a20 con 0x562306139620 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9c9a7fc640 10 client.4806 handle_command_reply: tid=0 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.150+0000 7f9cc49fd740 1 client.4806 shutdown 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2066237958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 msgr2=0x5623060d54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060d54f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0000c00 tx=0x7f9cb002e000 comp rx=0 tx=0).stop 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2066237958 shutdown_connections 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623060a29c0 0x562306133220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562306080620 0x5623060d54f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:6833/153100987,v1:172.21.15.122:6835/153100987] conn(0x562306139620 0x562306139a00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 --2- 172.21.15.71:0/2066237958 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56230607f970 0x5623060b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9c9a7fc640 0 client.4806 ms_handle_reset on v2:172.21.15.122:6833/153100987 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2066237958 >> 172.21.15.71:0/2066237958 conn(0x56230607f590 msgr2=0x5623060a0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2066237958 shutdown_connections 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 1 -- 172.21.15.71:0/2066237958 wait complete. 2024-04-16T15:40:50.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc49fd740 20 client.4806 trim_cache size 0 max 16384 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "c5697e20-5e32-4f13-b48e-2f7b8429c0b4", 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4296, 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 25.62971748, 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 9, 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 24, 2024-04-16T15:40:50.157 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 24, 2024-04-16T15:40:50.158 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 32.795333612999997 2024-04-16T15:40:50.158 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 -- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9ca806c8f0 msgr2=0x7f9ca806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9ca806c8f0 0x7f9ca806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc1e9790 tx=0x7f9cac009190 comp rx=0 tx=0).stop 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 -- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0c3570 msgr2=0x7f9cbc1d6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0c3570 0x7f9cbc1d6c30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400c680 tx=0x7f9cb400cb40 comp rx=0 tx=0).stop 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 -- 172.21.15.71:0/3190085804 shutdown_connections 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f9ca806c8f0 0x7f9ca806edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cbc0c3570 0x7f9cbc1d6c30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9cbc0726c0 0x7f9cbc1dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 --2- 172.21.15.71:0/3190085804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cbc06cbb0 0x7f9cbc1ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 -- 172.21.15.71:0/3190085804 >> 172.21.15.71:0/3190085804 conn(0x7f9cbc0c45a0 msgr2=0x7f9cbc06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 -- 172.21.15.71:0/3190085804 shutdown_connections 2024-04-16T15:40:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.151+0000 7f9cc217c640 1 -- 172.21.15.71:0/3190085804 wait complete. 2024-04-16T15:40:50.161 INFO:teuthology.run_tasks:Running task sequential... 2024-04-16T15:40:50.172 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-16T15:40:50.172 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-16T15:40:50.172 INFO:teuthology.task.print:Enabling mgr modules 2024-04-16T15:40:50.173 INFO:teuthology.run_tasks:Running task kclient... 2024-04-16T15:40:50.215 INFO:tasks.kclient:Mounting kernel clients... 2024-04-16T15:40:50.216 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-04-16T15:40:50.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:50.216 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-04-16T15:40:50.232 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-16T15:40:50.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:50.232 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T15:40:50.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:50.301 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-04-16T15:40:50.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:50.398 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-04-16T15:40:50.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:50.418 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-04-16T15:40:50.492 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-04-16T15:40:50.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:40:50.494 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-16T15:40:50.494 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T15:40:50.494 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T15:40:50.494 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-16T15:40:50.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.600+0000 7f37e8214640 1 Processor -- start 2024-04-16T15:40:50.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.601+0000 7f37e8214640 1 -- start start 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.601+0000 7f37e8214640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e006fac0 0x7f37e00c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.601+0000 7f37e8214640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e00c27c0 con 0x7f37e0155dc0 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.601+0000 7f37e8214640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e00c2aa0 con 0x7f37e00ab290 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.601+0000 7f37e8214640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37e0073f20 con 0x7f37e006fac0 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e5fb0640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f37e0155dc0 0x7f37e01561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34806/0 (socket says 172.21.15.71:34806) 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e5fb0640 1 -- 172.21.15.71:0/385920998 learned_addr learned my addr 172.21.15.71:0/385920998 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e57af640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e006fac0 0x7f37e00c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3891601067 0 0) 0x7f37e00c27c0 con 0x7f37e0155dc0 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37cc005730 con 0x7f37e0155dc0 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 933704440 0 0) 0x7f37cc005730 con 0x7f37e0155dc0 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e006fac0 msgr2=0x7f37e00c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e006fac0 0x7f37e00c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 >> v1:172.21.15.122:6789/0 conn(0x7f37e00ab290 legacy=0x7f37e015d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e00c2d80 con 0x7f37e0155dc0 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f37dc001670 con 0x7f37e0155dc0 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 >> v1:172.21.15.71:6789/0 conn(0x7f37e0155dc0 legacy=0x7f37e01561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc005bf0 0x7f37cc006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc006a10 0x7f37cc006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37cc0073d0 0x7f37cc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc00e110 con 0x7f37cc005bf0 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc00e290 con 0x7f37cc006a10 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc00e410 con 0x7f37cc0073d0 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e5fb0640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc005bf0 0x7f37cc006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e57af640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc006a10 0x7f37cc006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e67b1640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37cc0073d0 0x7f37cc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e5fb0640 1 -- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37cc0073d0 msgr2=0x7f37cc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e5fb0640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37cc0073d0 0x7f37cc00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e5fb0640 1 -- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc006a10 msgr2=0x7f37cc006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.602+0000 7f37e5fb0640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc006a10 0x7f37cc006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.603+0000 7f37e5fb0640 1 -- 172.21.15.71:0/385920998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc005730 con 0x7f37cc005bf0 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.603+0000 7f37e67b1640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37cc0073d0 0x7f37cc00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.603+0000 7f37e5fb0640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc005bf0 0x7f37cc006010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c240 tx=0x7f37cc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ff3d65c23557ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.603+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc007d10 con 0x7f37cc005bf0 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.603+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc030070 con 0x7f37cc005bf0 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.603+0000 7f37e4fae640 1 -- 172.21.15.71:0/385920998 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc02cd40 con 0x7f37cc005bf0 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.604+0000 7f37e8214640 1 -- 172.21.15.71:0/385920998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc005bf0 msgr2=0x7f37cc006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.604+0000 7f37e8214640 1 --2- 172.21.15.71:0/385920998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc005bf0 0x7f37cc006010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c240 tx=0x7f37cc007a40 comp rx=0 tx=0).stop 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.604+0000 7f37e67b1640 1 -- 172.21.15.71:0/385920998 reap_dead start 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.604+0000 7f37e8214640 1 -- 172.21.15.71:0/385920998 shutdown_connections 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.604+0000 7f37e8214640 1 -- 172.21.15.71:0/385920998 >> 172.21.15.71:0/385920998 conn(0x7f37e00c3e10 msgr2=0x7f37e00c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.604+0000 7f37e8214640 1 -- 172.21.15.71:0/385920998 shutdown_connections 2024-04-16T15:40:50.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.604+0000 7f37e8214640 1 -- 172.21.15.71:0/385920998 wait complete. 2024-04-16T15:40:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.605+0000 7f37e8214640 1 Processor -- start 2024-04-16T15:40:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.605+0000 7f37e8214640 1 -- start start 2024-04-16T15:40:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e8214640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc005bf0 0x7f37e0145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e8214640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e01ea650 0x7f37e01428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e8214640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e01eaa90 0x7f37e0142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e8214640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37e006bd70 con 0x7f37e01eaa90 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e8214640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0156ab0 con 0x7f37cc005bf0 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e8214640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37e006bef0 con 0x7f37e01ea650 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e01eaa90 0x7f37e0142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e01eaa90 0x7f37e0142e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52880/0 (socket says 172.21.15.71:52880) 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 -- 172.21.15.71:0/1078008537 learned_addr learned my addr 172.21.15.71:0/1078008537 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e5fb0640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc005bf0 0x7f37e0145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e57af640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e01ea650 0x7f37e01428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 -- 172.21.15.71:0/1078008537 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e01ea650 msgr2=0x7f37e01428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e01ea650 0x7f37e01428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 -- 172.21.15.71:0/1078008537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc005bf0 msgr2=0x7f37e0145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc005bf0 0x7f37e0145740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 -- 172.21.15.71:0/1078008537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e0144ea0 con 0x7f37e01eaa90 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.606+0000 7f37e67b1640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e01eaa90 0x7f37e0142e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f37e00c3de0 tx=0x7f37d800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.607+0000 7f37d27fc640 1 -- 172.21.15.71:0/1078008537 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8003b80 con 0x7f37e01eaa90 2024-04-16T15:40:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.607+0000 7f37d27fc640 1 -- 172.21.15.71:0/1078008537 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d8003d40 con 0x7f37e01eaa90 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.607+0000 7f37d27fc640 1 -- 172.21.15.71:0/1078008537 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8006b50 con 0x7f37e01eaa90 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.607+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01eb000 con 0x7f37e01eaa90 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.607+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e0144ac0 con 0x7f37e01eaa90 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.608+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e0068d20 con 0x7f37e01eaa90 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.608+0000 7f37d27fc640 1 -- 172.21.15.71:0/1078008537 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37d801a020 con 0x7f37e01eaa90 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.608+0000 7f37d27fc640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37c406c920 0x7f37c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.609+0000 7f37d27fc640 1 -- 172.21.15.71:0/1078008537 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f37d808b2d0 con 0x7f37e01eaa90 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.609+0000 7f37e5fb0640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37c406c920 0x7f37c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.609+0000 7f37e5fb0640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37c406c920 0x7f37c406ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f37dc034000 tx=0x7f37dc009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:50.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.613+0000 7f37d27fc640 1 -- 172.21.15.71:0/1078008537 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37e0144ea0 con 0x7f37e01eaa90 2024-04-16T15:40:50.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.827+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f37e006bb50 con 0x7f37e01eaa90 2024-04-16T15:40:50.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.827+0000 7f37d27fc640 1 -- 172.21.15.71:0/1078008537 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f37d8059450 con 0x7f37e01eaa90 2024-04-16T15:40:50.826 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T15:40:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37c406c920 msgr2=0x7f37c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37c406c920 0x7f37c406ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f37dc034000 tx=0x7f37dc009030 comp rx=0 tx=0).stop 2024-04-16T15:40:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e01eaa90 msgr2=0x7f37e0142e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e01eaa90 0x7f37e0142e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f37e00c3de0 tx=0x7f37d800ead0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 shutdown_connections 2024-04-16T15:40:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37c406c920 0x7f37c406ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e01eaa90 0x7f37e0142e20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e01ea650 0x7f37e01428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 --2- 172.21.15.71:0/1078008537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37cc005bf0 0x7f37e0145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 >> 172.21.15.71:0/1078008537 conn(0x7f37e00c3e10 msgr2=0x7f37e00aca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 shutdown_connections 2024-04-16T15:40:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50.829+0000 7f37e8214640 1 -- 172.21.15.71:0/1078008537 wait complete. 2024-04-16T15:40:50.839 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-16T15:40:50.839 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T15:40:50.841 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-04-16T15:40:50.841 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-16T15:40:50.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T15:40:50.842 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-16T15:40:50.842 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T15:40:50.842 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T15:40:50.842 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-16T15:40:50.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:50.842 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-04-16T15:40:50.860 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T15:40:50.860 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T15:40:50.860 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T15:40:50.860 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:50.860 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-04-16T15:40:50.860 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:50.861 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:50.861 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:50.861 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-04-16T15:40:50.861 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:50.862 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:50.862 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-04-16T15:40:50.862 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:50.862 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:50.862 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T15:40:50.862 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:50.862 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-16T15:40:50.862 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42734sec preferred_lft 42734sec 2024-04-16T15:40:50.863 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2024-04-16T15:40:50.863 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:50.863 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T15:40:50.863 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:50.863 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:50.863 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2024-04-16T15:40:50.864 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2024-04-16T15:40:50.864 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2024-04-16T15:40:50.864 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T15:40:50.864 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:50.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:51.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:51.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:51.012 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T15:40:51.039 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-04-16T15:40:51.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:51.040 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-04-16T15:40:51.097 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-04-16T15:40:51.097 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T15:40:51.097 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-16T15:40:51.097 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-16T15:40:51.097 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T15:40:51.098 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:51.098 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:51.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-16T15:40:51.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-16T15:40:51.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-16T15:40:51.098 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:51.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:51.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:51.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:51.237 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-04-16T15:40:51.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:51.253 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-04-16T15:40:51.308 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:51.309 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:51.309 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T15:40:51.309 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-16T15:40:51.309 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:51.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:51.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:51.433 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-16T15:40:51.433 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:51.433 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:51.433 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-16T15:40:51.433 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T15:40:51.433 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-16T15:40:51.433 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-16T15:40:51.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-16T15:40:51.434 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:51.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:51.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:51.563 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:51.564 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-16T15:40:51.564 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-04-16T15:40:51.564 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T15:40:51.564 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-16T15:40:51.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:51.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:51.647 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T15:40:51.647 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-16T15:40:51.662 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-16T15:40:51.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T15:40:51.662 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-16T15:40:51.717 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-16T15:40:51.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:51.717 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-16T15:40:51.805 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-16T15:40:51.805 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-16T15:40:51.805 INFO:teuthology.orchestra.run.smithi071.stdout:invalid new device string format 2024-04-16T15:40:51.902 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-16T15:40:51.902 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-16T15:40:51.902 INFO:teuthology.orchestra.run.smithi071.stdout:invalid new device string format 2024-04-16T15:40:51.902 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.122:3300,172.21.15.122:3301" 2024-04-16T15:40:51.902 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-16T15:40:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-16T15:40:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-16T15:40:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:invalid new device string format 2024-04-16T15:40:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.122:3300,172.21.15.122:3301" 2024-04-16T15:40:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-16T15:40:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:6789,172.21.15.122:6789,172.21.15.122:6790" 2024-04-16T15:40:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.71:6789,172.21.15.122:6789,172.21.15.122:6790:/ 2024-04-16T15:40:51.918 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c5697e20-5e32-4f13-b48e-2f7b8429c0b4" will pass to kernel 2024-04-16T15:40:51.919 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-16T15:40:51.919 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-04-16T15:40:51.919 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-16T15:40:51.919 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-04-16T15:40:51.919 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-16T15:40:51.920 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-16T15:40:51.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:51.921 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-16T15:40:51.949 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T15:40:52.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T15:40:52.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:52.045 INFO:teuthology.orchestra.run.smithi071.stdout:client4815 2024-04-16T15:40:52.045 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:40:52.045 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:40:52.062 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T15:40:52.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T15:40:52.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:52.158 INFO:teuthology.orchestra.run.smithi071.stdout:client4815 2024-04-16T15:40:52.158 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T15:40:52.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T15:40:52.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:52.214 INFO:teuthology.orchestra.run.smithi071.stdout:client4815 2024-04-16T15:40:52.214 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c5697e20-5e32-4f13-b48e-2f7b8429c0b4.client4815/status') 2024-04-16T15:40:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:52.277 INFO:teuthology.orchestra.run.smithi071.stdout:instance: client.4815 (3)192.168.0.1:0/3755661947 2024-04-16T15:40:52.277 INFO:teuthology.orchestra.run.smithi071.stdout:blocklisted: false 2024-04-16T15:40:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records in 2024-04-16T15:40:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records out 2024-04-16T15:40:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:69 bytes copied, 2.0991e-05 s, 3.3 MB/s 2024-04-16T15:40:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:52.284 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-16T15:40:52.284 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T15:40:52.284 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-16T15:40:52.284 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-16T15:40:52.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.388+0000 7fc32060d640 1 Processor -- start 2024-04-16T15:40:52.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.388+0000 7fc32060d640 1 -- start start 2024-04-16T15:40:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc32060d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc318155dc0 0x7fc3180c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc32060d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fc31806fac0 0x7fc3181561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc32060d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc3181527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc32060d640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fc318073f20 con 0x7fc31806fac0 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc32060d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc318156ab0 con 0x7fc318155dc0 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc32060d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc3180c27c0 con 0x7fc3180ab290 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc31dba8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fc31806fac0 0x7fc3181561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc31dba8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fc31806fac0 0x7fc3181561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52894/0 (socket says 172.21.15.71:52894) 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.389+0000 7fc31dba8640 1 -- 172.21.15.71:0/101759330 learned_addr learned my addr 172.21.15.71:0/101759330 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31ebaa640 1 --2- 172.21.15.71:0/101759330 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc3181527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31e3a9640 1 --2- 172.21.15.71:0/101759330 >> v2:172.21.15.122:3300/0 conn(0x7fc318155dc0 0x7fc3180c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31dba8640 1 -- 172.21.15.71:0/101759330 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 msgr2=0x7fc3181527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31dba8640 1 --2- 172.21.15.71:0/101759330 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc3181527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31dba8640 1 -- 172.21.15.71:0/101759330 >> v2:172.21.15.122:3300/0 conn(0x7fc318155dc0 msgr2=0x7fc3180c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31dba8640 1 --2- 172.21.15.71:0/101759330 >> v2:172.21.15.122:3300/0 conn(0x7fc318155dc0 0x7fc3180c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31dba8640 1 -- 172.21.15.71:0/101759330 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3180c2aa0 con 0x7fc31806fac0 2024-04-16T15:40:52.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31e3a9640 1 --2- 172.21.15.71:0/101759330 >> v2:172.21.15.122:3300/0 conn(0x7fc318155dc0 0x7fc3180c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:52.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31ebaa640 1 --2- 172.21.15.71:0/101759330 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc3181527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:52.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31dba8640 1 --2- 172.21.15.71:0/101759330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc31806fac0 0x7fc3181561a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc31806c190 tx=0x7fc30c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870b300cc2d49f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.390+0000 7fc31d3a7640 1 -- 172.21.15.71:0/101759330 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c03b070 con 0x7fc31806fac0 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc31d3a7640 1 -- 172.21.15.71:0/101759330 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30c036440 con 0x7fc31806fac0 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc31d3a7640 1 -- 172.21.15.71:0/101759330 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c036810 con 0x7fc31806fac0 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 -- 172.21.15.71:0/101759330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc31806fac0 msgr2=0x7fc3181561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 --2- 172.21.15.71:0/101759330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc31806fac0 0x7fc3181561a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc31806c190 tx=0x7fc30c02e0e0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 -- 172.21.15.71:0/101759330 shutdown_connections 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 --2- 172.21.15.71:0/101759330 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc3181527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 --2- 172.21.15.71:0/101759330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc31806fac0 0x7fc3181561a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 --2- 172.21.15.71:0/101759330 >> v2:172.21.15.122:3300/0 conn(0x7fc318155dc0 0x7fc3180c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 -- 172.21.15.71:0/101759330 >> 172.21.15.71:0/101759330 conn(0x7fc3180c3e10 msgr2=0x7fc3180c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 -- 172.21.15.71:0/101759330 shutdown_connections 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.391+0000 7fc32060d640 1 -- 172.21.15.71:0/101759330 wait complete. 2024-04-16T15:40:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.392+0000 7fc32060d640 1 Processor -- start 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.392+0000 7fc32060d640 1 -- start start 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc32060d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc318072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc32060d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc318155dc0 0x7fc3181eb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc32060d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3181eb850 0x7fc3181e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc32060d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181566e0 con 0x7fc318155dc0 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc32060d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3180c18b0 con 0x7fc3181eb850 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc32060d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc318156860 con 0x7fc3180ab290 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc318155dc0 0x7fc3181eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc318155dc0 0x7fc3181eb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52902/0 (socket says 172.21.15.71:52902) 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 -- 172.21.15.71:0/1091204136 learned_addr learned my addr 172.21.15.71:0/1091204136 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31ebaa640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3181eb850 0x7fc3181e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31e3a9640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc318072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 -- 172.21.15.71:0/1091204136 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 msgr2=0x7fc318072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:52.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc318072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 -- 172.21.15.71:0/1091204136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3181eb850 msgr2=0x7fc3181e7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3181eb850 0x7fc3181e7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 -- 172.21.15.71:0/1091204136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31814db60 con 0x7fc318155dc0 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.393+0000 7fc31dba8640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc318155dc0 0x7fc3181eb310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc30c02e120 tx=0x7fc30c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.394+0000 7fc316ffd640 1 -- 172.21.15.71:0/1091204136 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c03b040 con 0x7fc318155dc0 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.394+0000 7fc316ffd640 1 -- 172.21.15.71:0/1091204136 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30c046070 con 0x7fc318155dc0 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.394+0000 7fc316ffd640 1 -- 172.21.15.71:0/1091204136 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c034bb0 con 0x7fc318155dc0 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.394+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3180c6670 con 0x7fc318155dc0 2024-04-16T15:40:52.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.394+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3180731c0 con 0x7fc318155dc0 2024-04-16T15:40:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.395+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc318068d20 con 0x7fc318155dc0 2024-04-16T15:40:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.395+0000 7fc316ffd640 1 -- 172.21.15.71:0/1091204136 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc30c04b460 con 0x7fc318155dc0 2024-04-16T15:40:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.396+0000 7fc316ffd640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc30406c8f0 0x7fc30406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.396+0000 7fc316ffd640 1 -- 172.21.15.71:0/1091204136 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc30c0b0d20 con 0x7fc318155dc0 2024-04-16T15:40:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.396+0000 7fc31e3a9640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc30406c8f0 0x7fc30406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:52.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.396+0000 7fc31e3a9640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc30406c8f0 0x7fc30406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc30800b7c0 tx=0x7fc308008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:52.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.400+0000 7fc316ffd640 1 -- 172.21.15.71:0/1091204136 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc30c049360 con 0x7fc318155dc0 2024-04-16T15:40:52.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.615+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc31806cd40 con 0x7fc318155dc0 2024-04-16T15:40:52.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.615+0000 7fc316ffd640 1 -- 172.21.15.71:0/1091204136 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fc30c07eea0 con 0x7fc318155dc0 2024-04-16T15:40:52.614 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc30406c8f0 msgr2=0x7fc30406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc30406c8f0 0x7fc30406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc30800b7c0 tx=0x7fc308008040 comp rx=0 tx=0).stop 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc318155dc0 msgr2=0x7fc3181eb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc318155dc0 0x7fc3181eb310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc30c02e120 tx=0x7fc30c0046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 shutdown_connections 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc30406c8f0 0x7fc30406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3181eb850 0x7fc3181e7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc318155dc0 0x7fc3181eb310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 --2- 172.21.15.71:0/1091204136 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3180ab290 0x7fc318072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:52.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 >> 172.21.15.71:0/1091204136 conn(0x7fc3180c3e10 msgr2=0x7fc3180c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:52.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 shutdown_connections 2024-04-16T15:40:52.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:52.617+0000 7fc32060d640 1 -- 172.21.15.71:0/1091204136 wait complete. 2024-04-16T15:40:52.626 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-16T15:40:52.627 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T15:40:52.627 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-04-16T15:40:52.627 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-16T15:40:52.627 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-16T15:40:52.627 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-16T15:40:52.627 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T15:40:52.627 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T15:40:52.627 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-16T15:40:52.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:52.627 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-04-16T15:40:52.647 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T15:40:52.647 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T15:40:52.647 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-04-16T15:40:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-16T15:40:52.649 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42739sec preferred_lft 42739sec 2024-04-16T15:40:52.649 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-04-16T15:40:52.649 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-16T15:40:52.649 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T15:40:52.649 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:52.649 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-16T15:40:52.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add name ceph-brx type bridge 2024-04-16T15:40:52.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr flush dev ceph-brx 2024-04-16T15:40:52.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set ceph-brx up 2024-04-16T15:40:52.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T15:40:52.649 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-16T15:40:52.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:52.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:52.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:52.876 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T15:40:52.906 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-04-16T15:40:52.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:52.907 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-04-16T15:40:52.980 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-04-16T15:40:52.980 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T15:40:52.980 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-16T15:40:52.980 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-16T15:40:52.980 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T15:40:52.981 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:52.981 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-16T15:40:52.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-16T15:40:52.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-16T15:40:52.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-16T15:40:52.981 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-16T15:40:53.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:53.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:53.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:53.292 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-04-16T15:40:53.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:53.308 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-04-16T15:40:53.364 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:53.364 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-16T15:40:53.365 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-16T15:40:53.365 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-16T15:40:53.365 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-16T15:40:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:53.469 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:40:53.472 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:40:53.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:53.507 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-16T15:40:53.507 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:53.507 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-16T15:40:53.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-16T15:40:53.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T15:40:53.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-16T15:40:53.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-16T15:40:53.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-16T15:40:53.508 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-16T15:40:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:53.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:53.713 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T15:40:53.713 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-16T15:40:53.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-04-16T15:40:53.714 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T15:40:53.714 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-16T15:40:53.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:53.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T15:40:53.799 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-16T15:40:53.813 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-16T15:40:53.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T15:40:53.814 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-16T15:40:53.870 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-16T15:40:53.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:53.870 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-16T15:40:53.957 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-16T15:40:53.957 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-16T15:40:53.957 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-04-16T15:40:54.061 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-16T15:40:54.061 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-16T15:40:54.061 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-04-16T15:40:54.061 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.122:3300,172.21.15.122:3301" 2024-04-16T15:40:54.062 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.71:3300,172.21.15.122:3300,172.21.15.122:3301" 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.71:6789,172.21.15.122:6789,172.21.15.122:6790" 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.71:6789,172.21.15.122:6789,172.21.15.122:6790:/ 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c5697e20-5e32-4f13-b48e-2f7b8429c0b4" will pass to kernel 2024-04-16T15:40:54.069 INFO:teuthology.orchestra.run.smithi122.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-16T15:40:54.070 INFO:teuthology.orchestra.run.smithi122.stdout: You just mounted a file system that supports labels which does not 2024-04-16T15:40:54.070 INFO:teuthology.orchestra.run.smithi122.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-16T15:40:54.070 INFO:teuthology.orchestra.run.smithi122.stdout: applications will generate AVC messages and not be allowed access to 2024-04-16T15:40:54.070 INFO:teuthology.orchestra.run.smithi122.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-16T15:40:54.071 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-16T15:40:54.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:40:54.071 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-16T15:40:54.100 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T15:40:54.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T15:40:54.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout:client4708 2024-04-16T15:40:54.197 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:40:54.197 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:40:54.213 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T15:40:54.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T15:40:54.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:54.310 INFO:teuthology.orchestra.run.smithi122.stdout:client4708 2024-04-16T15:40:54.311 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T15:40:54.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T15:40:54.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:54.367 INFO:teuthology.orchestra.run.smithi122.stdout:client4708 2024-04-16T15:40:54.367 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c5697e20-5e32-4f13-b48e-2f7b8429c0b4.client4708/status') 2024-04-16T15:40:54.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T15:40:54.430 INFO:teuthology.orchestra.run.smithi122.stdout:instance: client.4708 (3)192.168.0.1:0/891818987 2024-04-16T15:40:54.430 INFO:teuthology.orchestra.run.smithi122.stdout:blocklisted: false 2024-04-16T15:40:54.430 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2024-04-16T15:40:54.430 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2024-04-16T15:40:54.430 INFO:teuthology.orchestra.run.smithi122.stderr:68 bytes copied, 2.1633e-05 s, 3.1 MB/s 2024-04-16T15:40:54.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-16T15:40:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T15:40:54.438 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T15:40:54.467 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T15:40:54.467 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:40:54.467 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/crash/warn_recent_interval 0' 2024-04-16T15:40:54.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba0e58f640 1 Processor -- start 2024-04-16T15:40:54.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba0e58f640 1 -- start start 2024-04-16T15:40:54.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba0e58f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fba08153430 0x7fba080c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba0e58f640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fba080c7b70 0x7fba080c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba0e58f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba080c4150 con 0x7fba080c44c0 2024-04-16T15:40:54.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba0e58f640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fba08159c20 con 0x7fba080c7b70 2024-04-16T15:40:54.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba0e58f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fba0815a150 con 0x7fba08153430 2024-04-16T15:40:54.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba077fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fba080c7b70 0x7fba080c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba077fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fba080c7b70 0x7fba080c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52904/0 (socket says 172.21.15.71:52904) 2024-04-16T15:40:54.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.584+0000 7fba077fe640 1 -- 172.21.15.71:0/2623119678 learned_addr learned my addr 172.21.15.71:0/2623119678 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:54.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.585+0000 7fba077fe640 1 -- 172.21.15.71:0/2623119678 >> v2:172.21.15.122:3300/0 conn(0x7fba08153430 msgr2=0x7fba080c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.585+0000 7fba077fe640 1 --2- 172.21.15.71:0/2623119678 >> v2:172.21.15.122:3300/0 conn(0x7fba08153430 0x7fba080c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.585+0000 7fba077fe640 1 -- 172.21.15.71:0/2623119678 >> v1:172.21.15.122:6789/0 conn(0x7fba080c44c0 legacy=0x7fba080c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.585+0000 7fba077fe640 1 -- 172.21.15.71:0/2623119678 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba08159e70 con 0x7fba080c7b70 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.585+0000 7fba077fe640 1 --2- 172.21.15.71:0/2623119678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba080c7b70 0x7fba080c72e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba080ac220 tx=0x7fb9fc02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e153945ae716eaec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba06ffd640 1 -- 172.21.15.71:0/2623119678 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc038070 con 0x7fba080c7b70 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba06ffd640 1 -- 172.21.15.71:0/2623119678 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc02f050 con 0x7fba080c7b70 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba06ffd640 1 -- 172.21.15.71:0/2623119678 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc031970 con 0x7fba080c7b70 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba0e58f640 1 -- 172.21.15.71:0/2623119678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba080c7b70 msgr2=0x7fba080c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2623119678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba080c7b70 0x7fba080c72e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fba080ac220 tx=0x7fb9fc02b7f0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba0e58f640 1 -- 172.21.15.71:0/2623119678 shutdown_connections 2024-04-16T15:40:54.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2623119678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba080c7b70 0x7fba080c72e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2623119678 >> v2:172.21.15.122:3300/0 conn(0x7fba08153430 0x7fba080c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba0e58f640 1 -- 172.21.15.71:0/2623119678 >> 172.21.15.71:0/2623119678 conn(0x7fba0806c020 msgr2=0x7fba080c93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:54.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba0e58f640 1 -- 172.21.15.71:0/2623119678 shutdown_connections 2024-04-16T15:40:54.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.586+0000 7fba0e58f640 1 -- 172.21.15.71:0/2623119678 wait complete. 2024-04-16T15:40:54.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.587+0000 7fba0e58f640 1 Processor -- start 2024-04-16T15:40:54.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.588+0000 7fba0e58f640 1 -- start start 2024-04-16T15:40:54.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.588+0000 7fba0e58f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba080c44c0 0x7fba081df590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.588+0000 7fba0e58f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba080c7b70 0x7fba081e1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.588+0000 7fba0e58f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08153430 0x7fba081582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.588+0000 7fba0e58f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fba08153810 con 0x7fba08153430 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.588+0000 7fba0e58f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba080c3de0 con 0x7fba080c7b70 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.588+0000 7fba0e58f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fba08153990 con 0x7fba080c44c0 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08153430 0x7fba081582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08153430 0x7fba081582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52912/0 (socket says 172.21.15.71:52912) 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 -- 172.21.15.71:0/2983110914 learned_addr learned my addr 172.21.15.71:0/2983110914 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba077fe640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba080c7b70 0x7fba081e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba07fff640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba080c44c0 0x7fba081df590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 -- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba080c44c0 msgr2=0x7fba081df590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba080c44c0 0x7fba081df590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 -- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba080c7b70 msgr2=0x7fba081e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba080c7b70 0x7fba081e1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 -- 172.21.15.71:0/2983110914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba081df0b0 con 0x7fba08153430 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba077fe640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba080c7b70 0x7fba081e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0cb2c640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08153430 0x7fba081582a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b990 tx=0x7fba0000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba04ff9640 1 -- 172.21.15.71:0/2983110914 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00019070 con 0x7fba08153430 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba04ff9640 1 -- 172.21.15.71:0/2983110914 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0000cba0 con 0x7fba08153430 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba081e1ae0 con 0x7fba08153430 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba04ff9640 1 -- 172.21.15.71:0/2983110914 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00014440 con 0x7fba08153430 2024-04-16T15:40:54.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.589+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba081efac0 con 0x7fba08153430 2024-04-16T15:40:54.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.590+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba081ef600 con 0x7fba08153430 2024-04-16T15:40:54.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.591+0000 7fba04ff9640 1 -- 172.21.15.71:0/2983110914 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba0001c070 con 0x7fba08153430 2024-04-16T15:40:54.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.591+0000 7fba04ff9640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.591+0000 7fba04ff9640 1 -- 172.21.15.71:0/2983110914 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fba0008bb60 con 0x7fba08153430 2024-04-16T15:40:54.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.591+0000 7fba07fff640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.592+0000 7fba07fff640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9f406c8f0 0x7fb9f406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800b810 tx=0x7fb9f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:54.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.596+0000 7fba04ff9640 1 -- 172.21.15.71:0/2983110914 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba081ef600 con 0x7fba08153430 2024-04-16T15:40:54.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.771+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/crash/warn_recent_interval}] v 0) -- 0x7fba08075fe0 con 0x7fba08153430 2024-04-16T15:40:54.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.776+0000 7fba04ff9640 1 -- 172.21.15.71:0/2983110914 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/crash/warn_recent_interval}]=0 v10)=0 v10) ==== 128+0+0 (secure 0 0 0) 0x7fba00059ce0 con 0x7fba08153430 2024-04-16T15:40:54.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9f406c8f0 msgr2=0x7fb9f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9f406c8f0 0x7fb9f406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800b810 tx=0x7fb9f8008040 comp rx=0 tx=0).stop 2024-04-16T15:40:54.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08153430 msgr2=0x7fba081582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08153430 0x7fba081582a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b990 tx=0x7fba0000be50 comp rx=0 tx=0).stop 2024-04-16T15:40:54.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 shutdown_connections 2024-04-16T15:40:54.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb9f406c8f0 0x7fb9f406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08153430 0x7fba081582a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba080c7b70 0x7fba081e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 --2- 172.21.15.71:0/2983110914 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba080c44c0 0x7fba081df590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 >> 172.21.15.71:0/2983110914 conn(0x7fba0806c020 msgr2=0x7fba08155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:54.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 shutdown_connections 2024-04-16T15:40:54.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.778+0000 7fba0e58f640 1 -- 172.21.15.71:0/2983110914 wait complete. 2024-04-16T15:40:54.788 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-16T15:40:54.805 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:54.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.909+0000 7f4c2f8e4640 1 Processor -- start 2024-04-16T15:40:54.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.910+0000 7f4c2f8e4640 1 -- start start 2024-04-16T15:40:54.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.910+0000 7f4c2f8e4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4c281516d0 0x7f4c28153d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.910+0000 7f4c2f8e4640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c28070270 con 0x7f4c2806fb20 2024-04-16T15:40:54.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.910+0000 7f4c2f8e4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c28070550 con 0x7f4c280c56a0 2024-04-16T15:40:54.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.910+0000 7f4c2f8e4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4c28072470 con 0x7f4c281516d0 2024-04-16T15:40:54.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.910+0000 7f4c2de81640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f4c2806fb20 0x7f4c280c44e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34818/0 (socket says 172.21.15.71:34818) 2024-04-16T15:40:54.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.910+0000 7f4c2de81640 1 -- 172.21.15.71:0/711758610 learned_addr learned my addr 172.21.15.71:0/711758610 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:54.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292201140 0 0) 0x7f4c28070270 con 0x7f4c2806fb20 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2ce7f640 1 --2- 172.21.15.71:0/711758610 >> v2:172.21.15.122:3300/0 conn(0x7f4c281516d0 0x7f4c28153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c10003520 con 0x7f4c2806fb20 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 199171638 0 0) 0x7f4c10003520 con 0x7f4c2806fb20 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 >> v2:172.21.15.122:3300/0 conn(0x7f4c281516d0 msgr2=0x7f4c28153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 --2- 172.21.15.71:0/711758610 >> v2:172.21.15.122:3300/0 conn(0x7f4c281516d0 0x7f4c28153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 >> v1:172.21.15.122:6789/0 conn(0x7f4c280c56a0 legacy=0x7f4c28150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c28070830 con 0x7f4c2806fb20 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f4c20001670 con 0x7f4c2806fb20 2024-04-16T15:40:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 >> v1:172.21.15.71:6789/0 conn(0x7f4c2806fb20 legacy=0x7f4c280c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c100039b0 0x7f4c10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c10004800 0x7f4c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c100051c0 0x7f4c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1000c130 con 0x7f4c10004800 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1000c2e0 con 0x7f4c100051c0 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1000c4e0 con 0x7f4c100039b0 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2ce7f640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c10004800 0x7f4c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2d680640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c100039b0 0x7f4c10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2de81640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c100051c0 0x7f4c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2ce7f640 1 -- 172.21.15.71:0/711758610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c100039b0 msgr2=0x7f4c10003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2ce7f640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c100039b0 0x7f4c10003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2ce7f640 1 -- 172.21.15.71:0/711758610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c100051c0 msgr2=0x7f4c1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2ce7f640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c100051c0 0x7f4c1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2de81640 1 -- 172.21.15.71:0/711758610 reap_dead start 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.911+0000 7f4c2ce7f640 1 -- 172.21.15.71:0/711758610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c10003520 con 0x7f4c10004800 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.912+0000 7f4c2ce7f640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c10004800 0x7f4c10004c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009ac0 tx=0x7f4c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee2032634f314f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.912+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c039070 con 0x7f4c10004800 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.912+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1c02c6b0 con 0x7f4c10004800 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.912+0000 7f4c27fff640 1 -- 172.21.15.71:0/711758610 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c009690 con 0x7f4c10004800 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.913+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/711758610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c10004800 msgr2=0x7f4c10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.913+0000 7f4c2f8e4640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c10004800 0x7f4c10004c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009ac0 tx=0x7f4c10005830 comp rx=0 tx=0).stop 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.913+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/711758610 shutdown_connections 2024-04-16T15:40:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.913+0000 7f4c2f8e4640 1 --2- 172.21.15.71:0/711758610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c10004800 0x7f4c10004c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.913+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/711758610 >> 172.21.15.71:0/711758610 conn(0x7f4c280ab2d0 msgr2=0x7f4c280ab6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:54.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.913+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/711758610 shutdown_connections 2024-04-16T15:40:54.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.913+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/711758610 wait complete. 2024-04-16T15:40:54.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.914+0000 7f4c2f8e4640 1 Processor -- start 2024-04-16T15:40:54.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.914+0000 7f4c2f8e4640 1 -- start start 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2f8e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c10004800 0x7f4c281ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2f8e4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c280c2950 0x7f4c280c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2f8e4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c281eb5b0 0x7f4c281eba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2f8e4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c28070b10 con 0x7f4c280c2950 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2f8e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c280c4050 con 0x7f4c10004800 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2f8e4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4c280c5a80 con 0x7f4c281eb5b0 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c280c2950 0x7f4c280c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c280c2950 0x7f4c280c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52922/0 (socket says 172.21.15.71:52922) 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 -- 172.21.15.71:0/72351085 learned_addr learned my addr 172.21.15.71:0/72351085 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2d680640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c10004800 0x7f4c281ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2de81640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c281eb5b0 0x7f4c281eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 -- 172.21.15.71:0/72351085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c281eb5b0 msgr2=0x7f4c281eba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c281eb5b0 0x7f4c281eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 -- 172.21.15.71:0/72351085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c10004800 msgr2=0x7f4c281ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c10004800 0x7f4c281ef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 -- 172.21.15.71:0/72351085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c281de200 con 0x7f4c280c2950 2024-04-16T15:40:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2ce7f640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c280c2950 0x7f4c280c2d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009ac0 tx=0x7f4c1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c25ffb640 1 -- 172.21.15.71:0/72351085 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c039050 con 0x7f4c280c2950 2024-04-16T15:40:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c25ffb640 1 -- 172.21.15.71:0/72351085 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1c045070 con 0x7f4c280c2950 2024-04-16T15:40:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c281ec240 con 0x7f4c280c2950 2024-04-16T15:40:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.915+0000 7f4c25ffb640 1 -- 172.21.15.71:0/72351085 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0044e0 con 0x7f4c280c2950 2024-04-16T15:40:54.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.916+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c281dd9f0 con 0x7f4c280c2950 2024-04-16T15:40:54.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.917+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c28068d20 con 0x7f4c280c2950 2024-04-16T15:40:54.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.917+0000 7f4c25ffb640 1 -- 172.21.15.71:0/72351085 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c1c003a50 con 0x7f4c280c2950 2024-04-16T15:40:54.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.918+0000 7f4c25ffb640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4c1406c8f0 0x7f4c1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:54.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.918+0000 7f4c25ffb640 1 -- 172.21.15.71:0/72351085 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4c1c0ada30 con 0x7f4c280c2950 2024-04-16T15:40:54.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.918+0000 7f4c2d680640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4c1406c8f0 0x7f4c1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.918+0000 7f4c2d680640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4c1406c8f0 0x7f4c1406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c18000f80 tx=0x7f4c18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:54.922+0000 7f4c25ffb640 1 -- 172.21.15.71:0/72351085 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c1c04e020 con 0x7f4c280c2950 2024-04-16T15:40:55.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.098+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c280af790 con 0x7f4c280c2950 2024-04-16T15:40:55.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.099+0000 7f4c25ffb640 1 -- 172.21.15.71:0/72351085 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f4c1c07bbb0 con 0x7f4c280c2950 2024-04-16T15:40:55.098 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:55.098 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4c1406c8f0 msgr2=0x7f4c1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4c1406c8f0 0x7f4c1406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c18000f80 tx=0x7f4c18009040 comp rx=0 tx=0).stop 2024-04-16T15:40:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c280c2950 msgr2=0x7f4c280c2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c280c2950 0x7f4c280c2d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009ac0 tx=0x7f4c1c004730 comp rx=0 tx=0).stop 2024-04-16T15:40:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 shutdown_connections 2024-04-16T15:40:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4c1406c8f0 0x7f4c1406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c281eb5b0 0x7f4c281eba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c280c2950 0x7f4c280c2d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 --2- 172.21.15.71:0/72351085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c10004800 0x7f4c281ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 >> 172.21.15.71:0/72351085 conn(0x7f4c280ab2d0 msgr2=0x7f4c280c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 shutdown_connections 2024-04-16T15:40:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.101+0000 7f4c2f8e4640 1 -- 172.21.15.71:0/72351085 wait complete. 2024-04-16T15:40:55.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:40:55.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:55.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.218+0000 7fe045590640 1 Processor -- start 2024-04-16T15:40:55.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.218+0000 7fe045590640 1 -- start start 2024-04-16T15:40:55.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe045590640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe0401533b0 0x7fe04006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe045590640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe04006f4d0 0x7fe04015d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe045590640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0400c3d60 con 0x7fe04006c420 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe045590640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fe040152f20 con 0x7fe0401533b0 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe045590640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe0401528d0 con 0x7fe04006f4d0 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe03e7fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe0401533b0 0x7fe04006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe03e7fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe0401533b0 0x7fe04006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52930/0 (socket says 172.21.15.71:52930) 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe03e7fc640 1 -- 172.21.15.71:0/2639435081 learned_addr learned my addr 172.21.15.71:0/2639435081 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.219+0000 7fe03f7fe640 1 --2- 172.21.15.71:0/2639435081 >> v2:172.21.15.122:3300/0 conn(0x7fe04006f4d0 0x7fe04015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.220+0000 7fe03e7fc640 1 -- 172.21.15.71:0/2639435081 >> v2:172.21.15.122:3300/0 conn(0x7fe04006f4d0 msgr2=0x7fe04015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.220+0000 7fe03e7fc640 1 --2- 172.21.15.71:0/2639435081 >> v2:172.21.15.122:3300/0 conn(0x7fe04006f4d0 0x7fe04015d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.220+0000 7fe03e7fc640 1 -- 172.21.15.71:0/2639435081 >> v1:172.21.15.122:6789/0 conn(0x7fe04006c420 legacy=0x7fe0401567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.220+0000 7fe03e7fc640 1 -- 172.21.15.71:0/2639435081 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe040152bb0 con 0x7fe0401533b0 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.220+0000 7fe03e7fc640 1 --2- 172.21.15.71:0/2639435081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe04006c800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe034009870 tx=0x7fe03402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc72fdd75128cac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.220+0000 7fe03dffb640 1 -- 172.21.15.71:0/2639435081 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034034070 con 0x7fe0401533b0 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.220+0000 7fe03dffb640 1 -- 172.21.15.71:0/2639435081 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe034030860 con 0x7fe0401533b0 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.220+0000 7fe03dffb640 1 -- 172.21.15.71:0/2639435081 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034030c30 con 0x7fe0401533b0 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.221+0000 7fe045590640 1 -- 172.21.15.71:0/2639435081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 msgr2=0x7fe04006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.221+0000 7fe045590640 1 --2- 172.21.15.71:0/2639435081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe04006c800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe034009870 tx=0x7fe03402b9a0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.221+0000 7fe045590640 1 -- 172.21.15.71:0/2639435081 shutdown_connections 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.221+0000 7fe045590640 1 --2- 172.21.15.71:0/2639435081 >> v2:172.21.15.122:3300/0 conn(0x7fe04006f4d0 0x7fe04015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.221+0000 7fe045590640 1 --2- 172.21.15.71:0/2639435081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe04006c800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.221+0000 7fe045590640 1 -- 172.21.15.71:0/2639435081 >> 172.21.15.71:0/2639435081 conn(0x7fe04006b930 msgr2=0x7fe04006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:55.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.221+0000 7fe045590640 1 -- 172.21.15.71:0/2639435081 shutdown_connections 2024-04-16T15:40:55.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.221+0000 7fe045590640 1 -- 172.21.15.71:0/2639435081 wait complete. 2024-04-16T15:40:55.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.222+0000 7fe045590640 1 Processor -- start 2024-04-16T15:40:55.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.222+0000 7fe045590640 1 -- start start 2024-04-16T15:40:55.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe045590640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe04006c420 0x7fe0401e7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe045590640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe04006f4d0 0x7fe0401e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe045590640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe040144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe045590640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe0400c4040 con 0x7fe0401533b0 2024-04-16T15:40:55.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe045590640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe04014cc80 con 0x7fe04006f4d0 2024-04-16T15:40:55.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe045590640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe0400c39e0 con 0x7fe04006c420 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe040144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe040144bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52938/0 (socket says 172.21.15.71:52938) 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 -- 172.21.15.71:0/1079414129 learned_addr learned my addr 172.21.15.71:0/1079414129 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03e7fc640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe04006f4d0 0x7fe0401e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03effd640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe04006c420 0x7fe0401e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 -- 172.21.15.71:0/1079414129 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe04006c420 msgr2=0x7fe0401e7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe04006c420 0x7fe0401e7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 -- 172.21.15.71:0/1079414129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe04006f4d0 msgr2=0x7fe0401e7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe04006f4d0 0x7fe0401e7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 -- 172.21.15.71:0/1079414129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0401e80c0 con 0x7fe0401533b0 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.223+0000 7fe03f7fe640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe040144bb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe03800e960 tx=0x7fe03800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.224+0000 7fe0277fe640 1 -- 172.21.15.71:0/1079414129 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038017070 con 0x7fe0401533b0 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.224+0000 7fe0277fe640 1 -- 172.21.15.71:0/1079414129 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03800c980 con 0x7fe0401533b0 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.224+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04014f7a0 con 0x7fe0401533b0 2024-04-16T15:40:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.224+0000 7fe0277fe640 1 -- 172.21.15.71:0/1079414129 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03800cd70 con 0x7fe0401533b0 2024-04-16T15:40:55.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.224+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0400aaa70 con 0x7fe0401533b0 2024-04-16T15:40:55.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.225+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe040068be0 con 0x7fe0401533b0 2024-04-16T15:40:55.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.225+0000 7fe0277fe640 1 -- 172.21.15.71:0/1079414129 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe03801a070 con 0x7fe0401533b0 2024-04-16T15:40:55.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.226+0000 7fe0277fe640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe02c06c8f0 0x7fe02c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.226+0000 7fe03effd640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe02c06c8f0 0x7fe02c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.226+0000 7fe0277fe640 1 -- 172.21.15.71:0/1079414129 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe03808be00 con 0x7fe0401533b0 2024-04-16T15:40:55.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.226+0000 7fe03effd640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe02c06c8f0 0x7fe02c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe04014ad50 tx=0x7fe030009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.230+0000 7fe0277fe640 1 -- 172.21.15.71:0/1079414129 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe038059f80 con 0x7fe0401533b0 2024-04-16T15:40:55.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.404+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe0400c37c0 con 0x7fe0401533b0 2024-04-16T15:40:55.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.405+0000 7fe0277fe640 1 -- 172.21.15.71:0/1079414129 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fe038059880 con 0x7fe0401533b0 2024-04-16T15:40:55.404 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:55.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe02c06c8f0 msgr2=0x7fe02c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe02c06c8f0 0x7fe02c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe04014ad50 tx=0x7fe030009040 comp rx=0 tx=0).stop 2024-04-16T15:40:55.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 msgr2=0x7fe040144bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe040144bb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe03800e960 tx=0x7fe03800ee20 comp rx=0 tx=0).stop 2024-04-16T15:40:55.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 shutdown_connections 2024-04-16T15:40:55.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe02c06c8f0 0x7fe02c06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0401533b0 0x7fe040144bb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe04006f4d0 0x7fe0401e7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 --2- 172.21.15.71:0/1079414129 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe04006c420 0x7fe0401e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 >> 172.21.15.71:0/1079414129 conn(0x7fe04006b930 msgr2=0x7fe04014bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:55.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 shutdown_connections 2024-04-16T15:40:55.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.407+0000 7fe045590640 1 -- 172.21.15.71:0/1079414129 wait complete. 2024-04-16T15:40:55.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:40:55.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.525+0000 7fb803655640 1 Processor -- start 2024-04-16T15:40:55.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.525+0000 7fb803655640 1 -- start start 2024-04-16T15:40:55.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb803655640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb7fc1531c0 0x7fb7fc0c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb803655640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7fc0c8590 con 0x7fb7fc06d9e0 2024-04-16T15:40:55.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb803655640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7fc0c8870 con 0x7fb7fc0c7fd0 2024-04-16T15:40:55.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb803655640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb7fc06d1e0 con 0x7fb7fc1531c0 2024-04-16T15:40:55.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb801bf2640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fb7fc06d9e0 0x7fb7fc0c5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34820/0 (socket says 172.21.15.71:34820) 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb801bf2640 1 -- 172.21.15.71:0/870003656 learned_addr learned my addr 172.21.15.71:0/870003656 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb8013f1640 1 --2- 172.21.15.71:0/870003656 >> v2:172.21.15.122:3300/0 conn(0x7fb7fc1531c0 0x7fb7fc0c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb7f3fff640 1 -- 172.21.15.71:0/870003656 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2338622100 0 0) 0x7fb7fc0c8590 con 0x7fb7fc06d9e0 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb7f3fff640 1 -- 172.21.15.71:0/870003656 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7e8003520 con 0x7fb7fc06d9e0 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb7f3fff640 1 -- 172.21.15.71:0/870003656 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3048598000 0 0) 0x7fb7fc0c8870 con 0x7fb7fc0c7fd0 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb7f3fff640 1 -- 172.21.15.71:0/870003656 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7fc0c8590 con 0x7fb7fc0c7fd0 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb8013f1640 1 -- 172.21.15.71:0/870003656 >> v1:172.21.15.122:6789/0 conn(0x7fb7fc0c7fd0 legacy=0x7fb7fc1566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb8013f1640 1 -- 172.21.15.71:0/870003656 >> v1:172.21.15.71:6789/0 conn(0x7fb7fc06d9e0 legacy=0x7fb7fc0c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.526+0000 7fb8013f1640 1 -- 172.21.15.71:0/870003656 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc06d4c0 con 0x7fb7fc1531c0 2024-04-16T15:40:55.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.527+0000 7fb8013f1640 1 --2- 172.21.15.71:0/870003656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc0c9610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80097a0 tx=0x7fb7f802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a64575099b12425 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.527+0000 7fb7f3fff640 1 -- 172.21.15.71:0/870003656 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80315a0 con 0x7fb7fc1531c0 2024-04-16T15:40:55.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb7f3fff640 1 -- 172.21.15.71:0/870003656 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f8037070 con 0x7fb7fc1531c0 2024-04-16T15:40:55.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb7f3fff640 1 -- 172.21.15.71:0/870003656 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8031960 con 0x7fb7fc1531c0 2024-04-16T15:40:55.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb803655640 1 -- 172.21.15.71:0/870003656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1531c0 msgr2=0x7fb7fc0c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb803655640 1 --2- 172.21.15.71:0/870003656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc0c9610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80097a0 tx=0x7fb7f802b840 comp rx=0 tx=0).stop 2024-04-16T15:40:55.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb803655640 1 -- 172.21.15.71:0/870003656 shutdown_connections 2024-04-16T15:40:55.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb803655640 1 --2- 172.21.15.71:0/870003656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc0c9610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb803655640 1 -- 172.21.15.71:0/870003656 >> 172.21.15.71:0/870003656 conn(0x7fb7fc0ad2f0 msgr2=0x7fb7fc0ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:55.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb803655640 1 -- 172.21.15.71:0/870003656 shutdown_connections 2024-04-16T15:40:55.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.528+0000 7fb803655640 1 -- 172.21.15.71:0/870003656 wait complete. 2024-04-16T15:40:55.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.529+0000 7fb803655640 1 Processor -- start 2024-04-16T15:40:55.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.529+0000 7fb803655640 1 -- start start 2024-04-16T15:40:55.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb803655640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc06d9e0 0x7fb7fc157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb803655640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7fc0c7fd0 0x7fb7fc0c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb803655640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc14b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb803655640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc153720 con 0x7fb7fc1531c0 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb803655640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc1535a0 con 0x7fb7fc06d9e0 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb803655640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb7fc06de10 con 0x7fb7fc0c7fd0 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb801bf2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc14b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb801bf2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc14b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52944/0 (socket says 172.21.15.71:52944) 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb801bf2640 1 -- 172.21.15.71:0/2773531427 learned_addr learned my addr 172.21.15.71:0/2773531427 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb8013f1640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc06d9e0 0x7fb7fc157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.530+0000 7fb800bf0640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7fc0c7fd0 0x7fb7fc0c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb801bf2640 1 -- 172.21.15.71:0/2773531427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7fc0c7fd0 msgr2=0x7fb7fc0c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb801bf2640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7fc0c7fd0 0x7fb7fc0c1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb801bf2640 1 -- 172.21.15.71:0/2773531427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc06d9e0 msgr2=0x7fb7fc157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb801bf2640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc06d9e0 0x7fb7fc157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb801bf2640 1 -- 172.21.15.71:0/2773531427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc159d30 con 0x7fb7fc1531c0 2024-04-16T15:40:55.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb801bf2640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc14b5b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400cb70 tx=0x7fb7f400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb7f1ffb640 1 -- 172.21.15.71:0/2773531427 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4017070 con 0x7fb7fc1531c0 2024-04-16T15:40:55.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb7f1ffb640 1 -- 172.21.15.71:0/2773531427 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f400ad40 con 0x7fb7fc1531c0 2024-04-16T15:40:55.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc0c27c0 con 0x7fb7fc1531c0 2024-04-16T15:40:55.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb7f1ffb640 1 -- 172.21.15.71:0/2773531427 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4012600 con 0x7fb7fc1531c0 2024-04-16T15:40:55.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.531+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc0c2220 con 0x7fb7fc1531c0 2024-04-16T15:40:55.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.532+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7fc074060 con 0x7fb7fc1531c0 2024-04-16T15:40:55.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.533+0000 7fb7f1ffb640 1 -- 172.21.15.71:0/2773531427 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7f401a070 con 0x7fb7fc1531c0 2024-04-16T15:40:55.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.533+0000 7fb7f1ffb640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb7e006c8f0 0x7fb7e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.533+0000 7fb8013f1640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb7e006c8f0 0x7fb7e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.533+0000 7fb7f1ffb640 1 -- 172.21.15.71:0/2773531427 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb7f408bb70 con 0x7fb7fc1531c0 2024-04-16T15:40:55.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.534+0000 7fb8013f1640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb7e006c8f0 0x7fb7e006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8036000 tx=0x7fb7f803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.539+0000 7fb7f1ffb640 1 -- 172.21.15.71:0/2773531427 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7f40596a0 con 0x7fb7fc1531c0 2024-04-16T15:40:55.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.715+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7fc068be0 con 0x7fb7fc1531c0 2024-04-16T15:40:55.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.715+0000 7fb7f1ffb640 1 -- 172.21.15.71:0/2773531427 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fb7f405d9a0 con 0x7fb7fc1531c0 2024-04-16T15:40:55.714 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:55.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb7e006c8f0 msgr2=0x7fb7e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb7e006c8f0 0x7fb7e006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8036000 tx=0x7fb7f803d040 comp rx=0 tx=0).stop 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1531c0 msgr2=0x7fb7fc14b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc14b5b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400cb70 tx=0x7fb7f400a430 comp rx=0 tx=0).stop 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 shutdown_connections 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb7e006c8f0 0x7fb7e006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc1531c0 0x7fb7fc14b5b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb7fc0c7fd0 0x7fb7fc0c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 --2- 172.21.15.71:0/2773531427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7fc06d9e0 0x7fb7fc157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 >> 172.21.15.71:0/2773531427 conn(0x7fb7fc0ad2f0 msgr2=0x7fb7fc154530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 shutdown_connections 2024-04-16T15:40:55.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.717+0000 7fb803655640 1 -- 172.21.15.71:0/2773531427 wait complete. 2024-04-16T15:40:55.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:40:55.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.835+0000 7f3c2c515640 1 Processor -- start 2024-04-16T15:40:55.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.835+0000 7f3c2c515640 1 -- start start 2024-04-16T15:40:55.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.836+0000 7f3c2c515640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3c2414da40 0x7f3c240ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.836+0000 7f3c2c515640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f3c240aca10 0x7f3c240c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.836+0000 7f3c2c515640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2414de20 con 0x7f3c240c3fb0 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.836+0000 7f3c2c515640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f3c240c9120 con 0x7f3c240aca10 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.836+0000 7f3c2c515640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f3c241548b0 con 0x7f3c2414da40 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.836+0000 7f3c29ab0640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f3c240aca10 0x7f3c240c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.836+0000 7f3c2aab2640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f3c240c3fb0 0x7f3c24156f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34822/0 (socket says 172.21.15.71:34822) 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.836+0000 7f3c2aab2640 1 -- 172.21.15.71:0/1475119879 learned_addr learned my addr 172.21.15.71:0/1475119879 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.837+0000 7f3c2a2b1640 1 --2- 172.21.15.71:0/1475119879 >> v2:172.21.15.122:3300/0 conn(0x7f3c2414da40 0x7f3c240ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.837+0000 7f3c29ab0640 1 -- 172.21.15.71:0/1475119879 >> v2:172.21.15.122:3300/0 conn(0x7f3c2414da40 msgr2=0x7f3c240ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.837+0000 7f3c29ab0640 1 --2- 172.21.15.71:0/1475119879 >> v2:172.21.15.122:3300/0 conn(0x7f3c2414da40 0x7f3c240ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.837+0000 7f3c29ab0640 1 -- 172.21.15.71:0/1475119879 >> v1:172.21.15.71:6789/0 conn(0x7f3c240c3fb0 legacy=0x7f3c24156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.837+0000 7f3c29ab0640 1 -- 172.21.15.71:0/1475119879 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c240ae840 con 0x7f3c240aca10 2024-04-16T15:40:55.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.837+0000 7f3c292af640 1 -- 172.21.15.71:0/1475119879 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894832297 0 0) 0x7f3c1c001490 con 0x7f3c240c3fb0 2024-04-16T15:40:55.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.837+0000 7f3c29ab0640 1 --2- 172.21.15.71:0/1475119879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c240aca10 0x7f3c240c78e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097a0 tx=0x7f3c1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7fcce22ecca200 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c292af640 1 -- 172.21.15.71:0/1475119879 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14035020 con 0x7f3c240aca10 2024-04-16T15:40:55.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c292af640 1 -- 172.21.15.71:0/1475119879 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c14031830 con 0x7f3c240aca10 2024-04-16T15:40:55.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c292af640 1 -- 172.21.15.71:0/1475119879 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14031c00 con 0x7f3c240aca10 2024-04-16T15:40:55.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c2c515640 1 -- 172.21.15.71:0/1475119879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c240aca10 msgr2=0x7f3c240c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c2c515640 1 --2- 172.21.15.71:0/1475119879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c240aca10 0x7f3c240c78e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097a0 tx=0x7f3c1402b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c2c515640 1 -- 172.21.15.71:0/1475119879 shutdown_connections 2024-04-16T15:40:55.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c2c515640 1 --2- 172.21.15.71:0/1475119879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c240aca10 0x7f3c240c78e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c2c515640 1 --2- 172.21.15.71:0/1475119879 >> v2:172.21.15.122:3300/0 conn(0x7f3c2414da40 0x7f3c240ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c2c515640 1 -- 172.21.15.71:0/1475119879 >> 172.21.15.71:0/1475119879 conn(0x7f3c240c9420 msgr2=0x7f3c240c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:55.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c2c515640 1 -- 172.21.15.71:0/1475119879 shutdown_connections 2024-04-16T15:40:55.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.838+0000 7f3c2c515640 1 -- 172.21.15.71:0/1475119879 wait complete. 2024-04-16T15:40:55.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.839+0000 7f3c2c515640 1 Processor -- start 2024-04-16T15:40:55.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.839+0000 7f3c2c515640 1 -- start start 2024-04-16T15:40:55.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2c515640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c240aca10 0x7f3c241e0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2c515640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c240c3fb0 0x7f3c24156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2c515640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c2414da40 0x7f3c24156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2c515640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3c240c4a10 con 0x7f3c2414da40 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2c515640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c240c73c0 con 0x7f3c240c3fb0 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2c515640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3c240c39f0 con 0x7f3c240aca10 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2aab2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c2414da40 0x7f3c24156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2aab2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c2414da40 0x7f3c24156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52962/0 (socket says 172.21.15.71:52962) 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2aab2640 1 -- 172.21.15.71:0/985546736 learned_addr learned my addr 172.21.15.71:0/985546736 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2a2b1640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c240aca10 0x7f3c241e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c29ab0640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c240c3fb0 0x7f3c24156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2aab2640 1 -- 172.21.15.71:0/985546736 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c240aca10 msgr2=0x7f3c241e0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2aab2640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c240aca10 0x7f3c241e0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2aab2640 1 -- 172.21.15.71:0/985546736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c240c3fb0 msgr2=0x7f3c24156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2aab2640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c240c3fb0 0x7f3c24156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.840+0000 7f3c2aab2640 1 -- 172.21.15.71:0/985546736 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2414a870 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.841+0000 7f3c2aab2640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c2414da40 0x7f3c24156aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c730 tx=0x7f3c1c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.841+0000 7f3c1affd640 1 -- 172.21.15.71:0/985546736 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c018070 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.841+0000 7f3c1affd640 1 -- 172.21.15.71:0/985546736 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c00ce40 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.841+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c241ea440 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.841+0000 7f3c1affd640 1 -- 172.21.15.71:0/985546736 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00a860 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.841+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c24155d60 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.842+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c241512f0 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.842+0000 7f3c1affd640 1 -- 172.21.15.71:0/985546736 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c1c01b070 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.843+0000 7f3c1affd640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3c0806c8f0 0x7f3c0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.843+0000 7f3c1affd640 1 -- 172.21.15.71:0/985546736 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3c1c08b400 con 0x7f3c2414da40 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.843+0000 7f3c2a2b1640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3c0806c8f0 0x7f3c0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:55.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.843+0000 7f3c2a2b1640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3c0806c8f0 0x7f3c0806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c24151210 tx=0x7f3c20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:55.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:55.847+0000 7f3c1affd640 1 -- 172.21.15.71:0/985546736 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c241512f0 con 0x7f3c2414da40 2024-04-16T15:40:56.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.023+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3c240c8540 con 0x7f3c2414da40 2024-04-16T15:40:56.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.023+0000 7f3c1affd640 1 -- 172.21.15.71:0/985546736 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f3c2414de20 con 0x7f3c2414da40 2024-04-16T15:40:56.022 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:40:56.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3c0806c8f0 msgr2=0x7f3c0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3c0806c8f0 0x7f3c0806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c24151210 tx=0x7f3c20008040 comp rx=0 tx=0).stop 2024-04-16T15:40:56.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c2414da40 msgr2=0x7f3c24156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c2414da40 0x7f3c24156aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c730 tx=0x7f3c1c00cbf0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 shutdown_connections 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3c0806c8f0 0x7f3c0806edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c2414da40 0x7f3c24156aa0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c240c3fb0 0x7f3c24156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 --2- 172.21.15.71:0/985546736 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3c240aca10 0x7f3c241e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 >> 172.21.15.71:0/985546736 conn(0x7f3c240c9420 msgr2=0x7f3c240c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 shutdown_connections 2024-04-16T15:40:56.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.025+0000 7f3c2c515640 1 -- 172.21.15.71:0/985546736 wait complete. 2024-04-16T15:40:56.035 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:40:56.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.142+0000 7f99613f4640 1 Processor -- start 2024-04-16T15:40:56.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.142+0000 7f99613f4640 1 -- start start 2024-04-16T15:40:56.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f99613f4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c151070 0x7f995c0ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f99613f4640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f995c0c1980 0x7f995c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f99613f4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f995c06ce80 0x7f995c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f99613f4640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f995c0aa850 con 0x7f995c0c1980 2024-04-16T15:40:56.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f99613f4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f995c0aab30 con 0x7f995c06ce80 2024-04-16T15:40:56.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f99613f4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f995c0aae10 con 0x7f995c151070 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995a7fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f995c0c1980 0x7f995c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995a7fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f995c0c1980 0x7f995c06c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52972/0 (socket says 172.21.15.71:52972) 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995a7fc640 1 -- 172.21.15.71:0/1578832168 learned_addr learned my addr 172.21.15.71:0/1578832168 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995b7fe640 1 --2- 172.21.15.71:0/1578832168 >> v2:172.21.15.122:3300/0 conn(0x7f995c06ce80 0x7f995c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995affd640 1 --2- 172.21.15.71:0/1578832168 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c151070 0x7f995c0ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995a7fc640 1 -- 172.21.15.71:0/1578832168 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c151070 msgr2=0x7f995c0ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995a7fc640 1 --2- 172.21.15.71:0/1578832168 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c151070 0x7f995c0ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995a7fc640 1 -- 172.21.15.71:0/1578832168 >> v2:172.21.15.122:3300/0 conn(0x7f995c06ce80 msgr2=0x7f995c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995a7fc640 1 --2- 172.21.15.71:0/1578832168 >> v2:172.21.15.122:3300/0 conn(0x7f995c06ce80 0x7f995c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.143+0000 7f995a7fc640 1 -- 172.21.15.71:0/1578832168 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c0ab0f0 con 0x7f995c0c1980 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.144+0000 7f995a7fc640 1 --2- 172.21.15.71:0/1578832168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c0c1980 0x7f995c06c940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f995c069530 tx=0x7f995002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dddad3651ce8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.144+0000 7f9959ffb640 1 -- 172.21.15.71:0/1578832168 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950035020 con 0x7f995c0c1980 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.144+0000 7f9959ffb640 1 -- 172.21.15.71:0/1578832168 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9950031870 con 0x7f995c0c1980 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.144+0000 7f9959ffb640 1 -- 172.21.15.71:0/1578832168 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950031c40 con 0x7f995c0c1980 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 -- 172.21.15.71:0/1578832168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c0c1980 msgr2=0x7f995c06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 --2- 172.21.15.71:0/1578832168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c0c1980 0x7f995c06c940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f995c069530 tx=0x7f995002b910 comp rx=0 tx=0).stop 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 -- 172.21.15.71:0/1578832168 shutdown_connections 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 --2- 172.21.15.71:0/1578832168 >> v2:172.21.15.122:3300/0 conn(0x7f995c06ce80 0x7f995c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 --2- 172.21.15.71:0/1578832168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c0c1980 0x7f995c06c940 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 --2- 172.21.15.71:0/1578832168 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c151070 0x7f995c0ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 -- 172.21.15.71:0/1578832168 >> 172.21.15.71:0/1578832168 conn(0x7f995c0c4200 msgr2=0x7f995c0c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 -- 172.21.15.71:0/1578832168 shutdown_connections 2024-04-16T15:40:56.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.145+0000 7f99613f4640 1 -- 172.21.15.71:0/1578832168 wait complete. 2024-04-16T15:40:56.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.146+0000 7f99613f4640 1 Processor -- start 2024-04-16T15:40:56.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.146+0000 7f99613f4640 1 -- start start 2024-04-16T15:40:56.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f99613f4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c06ce80 0x7f995c0c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f99613f4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995c0c1980 0x7f995c148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f99613f4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c151070 0x7f995c1489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f99613f4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f995c06edd0 con 0x7f995c151070 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f99613f4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f995c0acc60 con 0x7f995c0c1980 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f99613f4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f995c06ef50 con 0x7f995c06ce80 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c151070 0x7f995c1489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c151070 0x7f995c1489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52980/0 (socket says 172.21.15.71:52980) 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 -- 172.21.15.71:0/3202579472 learned_addr learned my addr 172.21.15.71:0/3202579472 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995a7fc640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995c0c1980 0x7f995c148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995affd640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c06ce80 0x7f995c0c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 -- 172.21.15.71:0/3202579472 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c06ce80 msgr2=0x7f995c0c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c06ce80 0x7f995c0c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 -- 172.21.15.71:0/3202579472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995c0c1980 msgr2=0x7f995c148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995c0c1980 0x7f995c148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 -- 172.21.15.71:0/3202579472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c0a7530 con 0x7f995c151070 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.147+0000 7f995b7fe640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c151070 0x7f995c1489b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f995400d600 tx=0x7f995400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.148+0000 7f99437fe640 1 -- 172.21.15.71:0/3202579472 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540125f0 con 0x7f995c151070 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.148+0000 7f99437fe640 1 -- 172.21.15.71:0/3202579472 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9954019070 con 0x7f995c151070 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.148+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c0a7810 con 0x7f995c151070 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.148+0000 7f99437fe640 1 -- 172.21.15.71:0/3202579472 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540129f0 con 0x7f995c151070 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.148+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c072af0 con 0x7f995c151070 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.149+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995c072cf0 con 0x7f995c151070 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.149+0000 7f99437fe640 1 -- 172.21.15.71:0/3202579472 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9954012b90 con 0x7f995c151070 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.149+0000 7f99437fe640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f994806c8f0 0x7f994806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.149+0000 7f99437fe640 1 -- 172.21.15.71:0/3202579472 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f995408c2b0 con 0x7f995c151070 2024-04-16T15:40:56.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.149+0000 7f995affd640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f994806c8f0 0x7f994806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.150+0000 7f995affd640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f994806c8f0 0x7f994806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f994c009940 tx=0x7f994c003cd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.154+0000 7f99437fe640 1 -- 172.21.15.71:0/3202579472 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f995c072cf0 con 0x7f995c151070 2024-04-16T15:40:56.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.326+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f995c068be0 con 0x7f995c151070 2024-04-16T15:40:56.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.326+0000 7f99437fe640 1 -- 172.21.15.71:0/3202579472 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f995405a3b0 con 0x7f995c151070 2024-04-16T15:40:56.325 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:56.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f994806c8f0 msgr2=0x7f994806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f994806c8f0 0x7f994806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f994c009940 tx=0x7f994c003cd0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c151070 msgr2=0x7f995c1489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c151070 0x7f995c1489b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f995400d600 tx=0x7f995400dac0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 shutdown_connections 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f994806c8f0 0x7f994806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c151070 0x7f995c1489b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995c0c1980 0x7f995c148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 --2- 172.21.15.71:0/3202579472 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995c06ce80 0x7f995c0c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 >> 172.21.15.71:0/3202579472 conn(0x7f995c0c4200 msgr2=0x7f995c1500b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 shutdown_connections 2024-04-16T15:40:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.328+0000 7f99613f4640 1 -- 172.21.15.71:0/3202579472 wait complete. 2024-04-16T15:40:56.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.441+0000 7f8714115640 1 Processor -- start 2024-04-16T15:40:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8714115640 1 -- start start 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8714115640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f870c159520 0x7f870c159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8714115640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f870c0c78d0 0x7f870c06df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8714115640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c06e470 0x7f870c0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8714115640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f870c0c9540 con 0x7f870c159520 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8714115640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f870c0ae4b0 con 0x7f870c0c78d0 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8714115640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f870c0c6940 con 0x7f870c06e470 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8711eb1640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f870c159520 0x7f870c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8711eb1640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f870c159520 0x7f870c159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52986/0 (socket says 172.21.15.71:52986) 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.442+0000 7f8711eb1640 1 -- 172.21.15.71:0/1687447166 learned_addr learned my addr 172.21.15.71:0/1687447166 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f87116b0640 1 --2- 172.21.15.71:0/1687447166 >> v2:172.21.15.122:3300/0 conn(0x7f870c0c78d0 0x7f870c06df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f87126b2640 1 --2- 172.21.15.71:0/1687447166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c06e470 0x7f870c0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f8711eb1640 1 -- 172.21.15.71:0/1687447166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c06e470 msgr2=0x7f870c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f8711eb1640 1 --2- 172.21.15.71:0/1687447166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c06e470 0x7f870c0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f8711eb1640 1 -- 172.21.15.71:0/1687447166 >> v2:172.21.15.122:3300/0 conn(0x7f870c0c78d0 msgr2=0x7f870c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f8711eb1640 1 --2- 172.21.15.71:0/1687447166 >> v2:172.21.15.122:3300/0 conn(0x7f870c0c78d0 0x7f870c06df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f8711eb1640 1 -- 172.21.15.71:0/1687447166 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c15a030 con 0x7f870c159520 2024-04-16T15:40:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f87116b0640 1 --2- 172.21.15.71:0/1687447166 >> v2:172.21.15.122:3300/0 conn(0x7f870c0c78d0 0x7f870c06df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.443+0000 7f8711eb1640 1 --2- 172.21.15.71:0/1687447166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c159520 0x7f870c159900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f870c0c66e0 tx=0x7f870802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4053b02d6b8afc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8710eaf640 1 -- 172.21.15.71:0/1687447166 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870802c950 con 0x7f870c159520 2024-04-16T15:40:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8710eaf640 1 -- 172.21.15.71:0/1687447166 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870802caf0 con 0x7f870c159520 2024-04-16T15:40:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8710eaf640 1 -- 172.21.15.71:0/1687447166 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87080343f0 con 0x7f870c159520 2024-04-16T15:40:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 -- 172.21.15.71:0/1687447166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c159520 msgr2=0x7f870c159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 --2- 172.21.15.71:0/1687447166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c159520 0x7f870c159900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f870c0c66e0 tx=0x7f870802bdb0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 -- 172.21.15.71:0/1687447166 shutdown_connections 2024-04-16T15:40:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 --2- 172.21.15.71:0/1687447166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c06e470 0x7f870c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 --2- 172.21.15.71:0/1687447166 >> v2:172.21.15.122:3300/0 conn(0x7f870c0c78d0 0x7f870c06df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 --2- 172.21.15.71:0/1687447166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c159520 0x7f870c159900 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 -- 172.21.15.71:0/1687447166 >> 172.21.15.71:0/1687447166 conn(0x7f870c0cb820 msgr2=0x7f870c0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 -- 172.21.15.71:0/1687447166 shutdown_connections 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.444+0000 7f8714115640 1 -- 172.21.15.71:0/1687447166 wait complete. 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.445+0000 7f8714115640 1 Processor -- start 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.445+0000 7f8714115640 1 -- start start 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f8714115640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f870c06e470 0x7f870c14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f8714115640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c0c78d0 0x7f870c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f8714115640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c159520 0x7f870c14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f8714115640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f870c0c9d50 con 0x7f870c0c78d0 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f8714115640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f870c0c9a70 con 0x7f870c06e470 2024-04-16T15:40:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f8714115640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f870c0b0cf0 con 0x7f870c159520 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87116b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c0c78d0 0x7f870c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87116b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c0c78d0 0x7f870c0ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52998/0 (socket says 172.21.15.71:52998) 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87116b0640 1 -- 172.21.15.71:0/3114007635 learned_addr learned my addr 172.21.15.71:0/3114007635 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f8711eb1640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f870c06e470 0x7f870c14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87126b2640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c159520 0x7f870c14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87116b0640 1 -- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c159520 msgr2=0x7f870c14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87116b0640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c159520 0x7f870c14dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87116b0640 1 -- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f870c06e470 msgr2=0x7f870c14ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87116b0640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f870c06e470 0x7f870c14ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f87116b0640 1 -- 172.21.15.71:0/3114007635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c0c2780 con 0x7f870c0c78d0 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.446+0000 7f8711eb1640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f870c06e470 0x7f870c14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.447+0000 7f87116b0640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c0c78d0 0x7f870c0ae900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00e7e0 tx=0x7f86fc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.447+0000 7f87027fc640 1 -- 172.21.15.71:0/3114007635 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00f040 con 0x7f870c0c78d0 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.447+0000 7f87027fc640 1 -- 172.21.15.71:0/3114007635 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86fc004030 con 0x7f870c0c78d0 2024-04-16T15:40:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.447+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c0c30c0 con 0x7f870c0c78d0 2024-04-16T15:40:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.447+0000 7f87027fc640 1 -- 172.21.15.71:0/3114007635 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc004400 con 0x7f870c0c78d0 2024-04-16T15:40:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.447+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1561f0 con 0x7f870c0c78d0 2024-04-16T15:40:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.448+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f870c074060 con 0x7f870c0c78d0 2024-04-16T15:40:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.448+0000 7f87027fc640 1 -- 172.21.15.71:0/3114007635 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86fc00c6d0 con 0x7f870c0c78d0 2024-04-16T15:40:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.449+0000 7f87027fc640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f86f006c8f0 0x7f86f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.449+0000 7f87027fc640 1 -- 172.21.15.71:0/3114007635 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f86fc08bc10 con 0x7f870c0c78d0 2024-04-16T15:40:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.449+0000 7f8711eb1640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f86f006c8f0 0x7f86f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.449+0000 7f8711eb1640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f86f006c8f0 0x7f86f006edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8708035000 tx=0x7f8708030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.453+0000 7f87027fc640 1 -- 172.21.15.71:0/3114007635 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86fc017020 con 0x7f870c0c78d0 2024-04-16T15:40:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.634+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f870c0bebc0 con 0x7f870c0c78d0 2024-04-16T15:40:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.635+0000 7f87027fc640 1 -- 172.21.15.71:0/3114007635 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f86fc059d90 con 0x7f870c0c78d0 2024-04-16T15:40:56.634 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:56.634 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:56.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f86f006c8f0 msgr2=0x7f86f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f86f006c8f0 0x7f86f006edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8708035000 tx=0x7f8708030000 comp rx=0 tx=0).stop 2024-04-16T15:40:56.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c0c78d0 msgr2=0x7f870c0ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c0c78d0 0x7f870c0ae900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00e7e0 tx=0x7f86fc00eca0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 shutdown_connections 2024-04-16T15:40:56.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f86f006c8f0 0x7f86f006edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f870c159520 0x7f870c14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f870c0c78d0 0x7f870c0ae900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 --2- 172.21.15.71:0/3114007635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f870c06e470 0x7f870c14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 >> 172.21.15.71:0/3114007635 conn(0x7f870c0cb820 msgr2=0x7f870c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:56.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 shutdown_connections 2024-04-16T15:40:56.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.637+0000 7f8714115640 1 -- 172.21.15.71:0/3114007635 wait complete. 2024-04-16T15:40:56.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:40:56.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:56.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.755+0000 7feddffdb640 1 Processor -- start 2024-04-16T15:40:56.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.755+0000 7feddffdb640 1 -- start start 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7feddffdb640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fedd8153320 0x7fedd80c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7feddffdb640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fedd80c8c10 0x7fedd80c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7feddffdb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd81564f0 0x7fedd8071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7feddffdb640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fedd80af4c0 con 0x7fedd8153320 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7feddffdb640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fedd8155d30 con 0x7fedd80c8c10 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7feddffdb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fedd80c3d50 con 0x7fedd81564f0 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedddd77640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fedd8153320 0x7fedd80c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedddd77640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fedd8153320 0x7fedd80c26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53014/0 (socket says 172.21.15.71:53014) 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedddd77640 1 -- 172.21.15.71:0/4133334782 learned_addr learned my addr 172.21.15.71:0/4133334782 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7feddd576640 1 --2- 172.21.15.71:0/4133334782 >> v2:172.21.15.122:3300/0 conn(0x7fedd80c8c10 0x7fedd80c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedde578640 1 --2- 172.21.15.71:0/4133334782 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd81564f0 0x7fedd8071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedddd77640 1 -- 172.21.15.71:0/4133334782 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd81564f0 msgr2=0x7fedd8071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedddd77640 1 --2- 172.21.15.71:0/4133334782 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd81564f0 0x7fedd8071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedddd77640 1 -- 172.21.15.71:0/4133334782 >> v2:172.21.15.122:3300/0 conn(0x7fedd80c8c10 msgr2=0x7fedd80c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedddd77640 1 --2- 172.21.15.71:0/4133334782 >> v2:172.21.15.122:3300/0 conn(0x7fedd80c8c10 0x7fedd80c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedddd77640 1 -- 172.21.15.71:0/4133334782 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd80c4030 con 0x7fedd8153320 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.756+0000 7fedde578640 1 --2- 172.21.15.71:0/4133334782 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd81564f0 0x7fedd8071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.757+0000 7fedddd77640 1 --2- 172.21.15.71:0/4133334782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd8153320 0x7fedd80c26e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fedd80c6c40 tx=0x7fedd402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bdf90bc95d87a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.757+0000 7feddcd75640 1 -- 172.21.15.71:0/4133334782 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4038070 con 0x7fedd8153320 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.757+0000 7feddcd75640 1 -- 172.21.15.71:0/4133334782 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd402f050 con 0x7fedd8153320 2024-04-16T15:40:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.757+0000 7feddcd75640 1 -- 172.21.15.71:0/4133334782 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4031970 con 0x7fedd8153320 2024-04-16T15:40:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 -- 172.21.15.71:0/4133334782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd8153320 msgr2=0x7fedd80c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 --2- 172.21.15.71:0/4133334782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd8153320 0x7fedd80c26e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fedd80c6c40 tx=0x7fedd402b7f0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 -- 172.21.15.71:0/4133334782 shutdown_connections 2024-04-16T15:40:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 --2- 172.21.15.71:0/4133334782 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd81564f0 0x7fedd8071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 --2- 172.21.15.71:0/4133334782 >> v2:172.21.15.122:3300/0 conn(0x7fedd80c8c10 0x7fedd80c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 --2- 172.21.15.71:0/4133334782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd8153320 0x7fedd80c26e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 -- 172.21.15.71:0/4133334782 >> 172.21.15.71:0/4133334782 conn(0x7fedd80c5db0 msgr2=0x7fedd80c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 -- 172.21.15.71:0/4133334782 shutdown_connections 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.758+0000 7feddffdb640 1 -- 172.21.15.71:0/4133334782 wait complete. 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.759+0000 7feddffdb640 1 Processor -- start 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.759+0000 7feddffdb640 1 -- start start 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddffdb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd80c8c10 0x7fedd8152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddffdb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd8153320 0x7fedd81ebfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddffdb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd81564f0 0x7fedd81ec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddffdb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8156940 con 0x7fedd81564f0 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddffdb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedd80c4bb0 con 0x7fedd80c8c10 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddffdb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fedd8156ac0 con 0x7fedd8153320 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd81564f0 0x7fedd81ec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd81564f0 0x7fedd81ec4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53024/0 (socket says 172.21.15.71:53024) 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 -- 172.21.15.71:0/1248750825 learned_addr learned my addr 172.21.15.71:0/1248750825 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedddd77640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd80c8c10 0x7fedd8152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddd576640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd8153320 0x7fedd81ebfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 -- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd8153320 msgr2=0x7fedd81ebfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd8153320 0x7fedd81ebfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 -- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd80c8c10 msgr2=0x7fedd8152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd80c8c10 0x7fedd8152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 -- 172.21.15.71:0/1248750825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd8149960 con 0x7fedd81564f0 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddd576640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd8153320 0x7fedd81ebfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedde578640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd81564f0 0x7fedd81ec4f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fedd000cf20 tx=0x7fedd000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedce7fc640 1 -- 172.21.15.71:0/1248750825 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd001a070 con 0x7fedd81564f0 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedce7fc640 1 -- 172.21.15.71:0/1248750825 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd000d840 con 0x7fedd81564f0 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd8149680 con 0x7fedd81564f0 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7fedce7fc640 1 -- 172.21.15.71:0/1248750825 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd000dc30 con 0x7fedd81564f0 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.760+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd814b8d0 con 0x7fedd81564f0 2024-04-16T15:40:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.761+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd80c33c0 con 0x7fedd81564f0 2024-04-16T15:40:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.762+0000 7fedce7fc640 1 -- 172.21.15.71:0/1248750825 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedd001d070 con 0x7fedd81564f0 2024-04-16T15:40:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.762+0000 7fedce7fc640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fedbc06c8f0 0x7fedbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.762+0000 7fedce7fc640 1 -- 172.21.15.71:0/1248750825 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fedd008c5f0 con 0x7fedd81564f0 2024-04-16T15:40:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.762+0000 7fedddd77640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fedbc06c8f0 0x7fedbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.763+0000 7fedddd77640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fedbc06c8f0 0x7fedbc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fedd4035000 tx=0x7fedd4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:56.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.767+0000 7fedce7fc640 1 -- 172.21.15.71:0/1248750825 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedd80c33c0 con 0x7fedd81564f0 2024-04-16T15:40:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.941+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fedd8069510 con 0x7fedd81564f0 2024-04-16T15:40:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.942+0000 7fedce7fc640 1 -- 172.21.15.71:0/1248750825 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fedd8069510 con 0x7fedd81564f0 2024-04-16T15:40:56.941 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fedbc06c8f0 msgr2=0x7fedbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fedbc06c8f0 0x7fedbc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fedd4035000 tx=0x7fedd4008030 comp rx=0 tx=0).stop 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd81564f0 msgr2=0x7fedd81ec4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd81564f0 0x7fedd81ec4f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fedd000cf20 tx=0x7fedd000a7d0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 shutdown_connections 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fedbc06c8f0 0x7fedbc06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd81564f0 0x7fedd81ec4f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fedd8153320 0x7fedd81ebfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 --2- 172.21.15.71:0/1248750825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd80c8c10 0x7fedd8152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 >> 172.21.15.71:0/1248750825 conn(0x7fedd80c5db0 msgr2=0x7fedd80c2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 shutdown_connections 2024-04-16T15:40:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:56.944+0000 7feddffdb640 1 -- 172.21.15.71:0/1248750825 wait complete. 2024-04-16T15:40:56.954 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:40:57.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.061+0000 7fce3a5a8640 1 Processor -- start 2024-04-16T15:40:57.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.061+0000 7fce3a5a8640 1 -- start start 2024-04-16T15:40:57.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.061+0000 7fce3a5a8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fce34150f80 0x7fce340c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.061+0000 7fce3a5a8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fce340c5b10 0x7fce340c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.061+0000 7fce3a5a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce340738e0 0x7fce341575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.061+0000 7fce3a5a8640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fce34157120 con 0x7fce34150f80 2024-04-16T15:40:57.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.061+0000 7fce3a5a8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fce34154890 con 0x7fce340c5b10 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.061+0000 7fce3a5a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fce34073300 con 0x7fce340738e0 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce33fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fce34150f80 0x7fce340c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce33fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fce34150f80 0x7fce340c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53028/0 (socket says 172.21.15.71:53028) 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce33fff640 1 -- 172.21.15.71:0/1433466757 learned_addr learned my addr 172.21.15.71:0/1433466757 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce2bfff640 1 --2- 172.21.15.71:0/1433466757 >> v2:172.21.15.122:3300/0 conn(0x7fce340c5b10 0x7fce340c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce38b45640 1 --2- 172.21.15.71:0/1433466757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce340738e0 0x7fce341575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce33fff640 1 -- 172.21.15.71:0/1433466757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce340738e0 msgr2=0x7fce341575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce33fff640 1 --2- 172.21.15.71:0/1433466757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce340738e0 0x7fce341575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce33fff640 1 -- 172.21.15.71:0/1433466757 >> v2:172.21.15.122:3300/0 conn(0x7fce340c5b10 msgr2=0x7fce340c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce33fff640 1 --2- 172.21.15.71:0/1433466757 >> v2:172.21.15.122:3300/0 conn(0x7fce340c5b10 0x7fce340c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce33fff640 1 -- 172.21.15.71:0/1433466757 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce340735e0 con 0x7fce34150f80 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce38b45640 1 --2- 172.21.15.71:0/1433466757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce340738e0 0x7fce341575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:57.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.062+0000 7fce2bfff640 1 --2- 172.21.15.71:0/1433466757 >> v2:172.21.15.122:3300/0 conn(0x7fce340c5b10 0x7fce340c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:57.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce33fff640 1 --2- 172.21.15.71:0/1433466757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce34150f80 0x7fce340c4d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fce340c6010 tx=0x7fce2402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f810488589d517ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce337fe640 1 -- 172.21.15.71:0/1433466757 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2402e070 con 0x7fce34150f80 2024-04-16T15:40:57.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce337fe640 1 -- 172.21.15.71:0/1433466757 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce24037070 con 0x7fce34150f80 2024-04-16T15:40:57.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce337fe640 1 -- 172.21.15.71:0/1433466757 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24031a00 con 0x7fce34150f80 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 -- 172.21.15.71:0/1433466757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce34150f80 msgr2=0x7fce340c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/1433466757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce34150f80 0x7fce340c4d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fce340c6010 tx=0x7fce2402b880 comp rx=0 tx=0).stop 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 -- 172.21.15.71:0/1433466757 shutdown_connections 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/1433466757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce340738e0 0x7fce341575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/1433466757 >> v2:172.21.15.122:3300/0 conn(0x7fce340c5b10 0x7fce340c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/1433466757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce34150f80 0x7fce340c4d80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 -- 172.21.15.71:0/1433466757 >> 172.21.15.71:0/1433466757 conn(0x7fce340c8470 msgr2=0x7fce340c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 -- 172.21.15.71:0/1433466757 shutdown_connections 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.063+0000 7fce3a5a8640 1 -- 172.21.15.71:0/1433466757 wait complete. 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce3a5a8640 1 Processor -- start 2024-04-16T15:40:57.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce3a5a8640 1 -- start start 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce3a5a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce340738e0 0x7fce3406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce3a5a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce340c5b10 0x7fce341e95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce3a5a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce34150f80 0x7fce341e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce3a5a8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fce34151360 con 0x7fce340738e0 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce3a5a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fce34154520 con 0x7fce340c5b10 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce3a5a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fce341514e0 con 0x7fce34150f80 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce33fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce340738e0 0x7fce3406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce33fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce340738e0 0x7fce3406d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53030/0 (socket says 172.21.15.71:53030) 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.065+0000 7fce33fff640 1 -- 172.21.15.71:0/3150779114 learned_addr learned my addr 172.21.15.71:0/3150779114 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce38b45640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce34150f80 0x7fce341e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce2bfff640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce340c5b10 0x7fce341e95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce33fff640 1 -- 172.21.15.71:0/3150779114 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce34150f80 msgr2=0x7fce341e9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce33fff640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce34150f80 0x7fce341e9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce33fff640 1 -- 172.21.15.71:0/3150779114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce340c5b10 msgr2=0x7fce341e95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce33fff640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce340c5b10 0x7fce341e95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce33fff640 1 -- 172.21.15.71:0/3150779114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3414ea00 con 0x7fce340738e0 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce33fff640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce340738e0 0x7fce3406d570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce2402b8c0 tx=0x7fce240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce317fa640 1 -- 172.21.15.71:0/3150779114 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24032880 con 0x7fce340738e0 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce317fa640 1 -- 172.21.15.71:0/3150779114 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce24032a40 con 0x7fce340738e0 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce317fa640 1 -- 172.21.15.71:0/3150779114 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24032e30 con 0x7fce340738e0 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce341ed580 con 0x7fce340738e0 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.066+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce34153560 con 0x7fce340738e0 2024-04-16T15:40:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.067+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce34062d80 con 0x7fce340738e0 2024-04-16T15:40:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.068+0000 7fce317fa640 1 -- 172.21.15.71:0/3150779114 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce2404c060 con 0x7fce340738e0 2024-04-16T15:40:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.068+0000 7fce317fa640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1406c8f0 0x7fce1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.068+0000 7fce317fa640 1 -- 172.21.15.71:0/3150779114 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fce240ad910 con 0x7fce340738e0 2024-04-16T15:40:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.068+0000 7fce2bfff640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1406c8f0 0x7fce1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.069+0000 7fce2bfff640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1406c8f0 0x7fce1406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce34145bb0 tx=0x7fce1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.072+0000 7fce317fa640 1 -- 172.21.15.71:0/3150779114 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce34062d80 con 0x7fce340738e0 2024-04-16T15:40:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.246+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fce3406ba70 con 0x7fce340738e0 2024-04-16T15:40:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.247+0000 7fce317fa640 1 -- 172.21.15.71:0/3150779114 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fce24040070 con 0x7fce340738e0 2024-04-16T15:40:57.246 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1406c8f0 msgr2=0x7fce1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1406c8f0 0x7fce1406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce34145bb0 tx=0x7fce1c008040 comp rx=0 tx=0).stop 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce340738e0 msgr2=0x7fce3406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce340738e0 0x7fce3406d570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fce2402b8c0 tx=0x7fce240046c0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 shutdown_connections 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1406c8f0 0x7fce1406edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce34150f80 0x7fce341e9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce340c5b10 0x7fce341e95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 --2- 172.21.15.71:0/3150779114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce340738e0 0x7fce3406d570 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 >> 172.21.15.71:0/3150779114 conn(0x7fce340c8470 msgr2=0x7fce340c3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 shutdown_connections 2024-04-16T15:40:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.249+0000 7fce3a5a8640 1 -- 172.21.15.71:0/3150779114 wait complete. 2024-04-16T15:40:57.259 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:40:57.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.367+0000 7f18dff09640 1 Processor -- start 2024-04-16T15:40:57.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.367+0000 7f18dff09640 1 -- start start 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dff09640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 0x7f18d80c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dff09640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18d8155d30 con 0x7f18d81564f0 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dff09640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18d80c3d50 con 0x7f18d8153320 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dff09640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f18d80af4c0 con 0x7f18d80c8c10 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18de4a6640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f18d81564f0 0x7f18d8071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34838/0 (socket says 172.21.15.71:34838) 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18de4a6640 1 -- 172.21.15.71:0/466076631 learned_addr learned my addr 172.21.15.71:0/466076631 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3059346757 0 0) 0x7f18d8155d30 con 0x7f18d81564f0 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18c4003520 con 0x7f18d81564f0 2024-04-16T15:40:57.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dd4a4640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 0x7f18d80c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1860945898 0 0) 0x7f18c4003520 con 0x7f18d81564f0 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 msgr2=0x7f18d80c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 0x7f18d80c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 >> v1:172.21.15.122:6789/0 conn(0x7f18d8153320 legacy=0x7f18d80c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d80c4030 con 0x7f18d81564f0 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f18d00019f0 con 0x7f18d81564f0 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 >> v1:172.21.15.71:6789/0 conn(0x7f18d81564f0 legacy=0x7f18d8071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c40039e0 0x7f18c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 0x7f18c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18c4004d90 0x7f18c4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18c400bc10 con 0x7f18c4004d90 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18c400bdc0 con 0x7f18c40039e0 2024-04-16T15:40:57.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.368+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f18c400bfc0 con 0x7f18d80c8c10 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18dd4a4640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 0x7f18c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18ddca5640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c40039e0 0x7f18c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18de4a6640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18c4004d90 0x7f18c4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18de4a6640 1 -- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 msgr2=0x7f18c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18de4a6640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 0x7f18c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18de4a6640 1 -- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c40039e0 msgr2=0x7f18c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18de4a6640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c40039e0 0x7f18c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18de4a6640 1 -- 172.21.15.71:0/466076631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c4003520 con 0x7f18c4004d90 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18dd4a4640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d80c8c10 0x7f18c4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.369+0000 7f18ddca5640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c40039e0 0x7f18c4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18de4a6640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18c4004d90 0x7f18c4005210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18d000a810 tx=0x7f18c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce88f6f1ca460f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d000a460 con 0x7f18c4004d90 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d0034630 con 0x7f18c4004d90 2024-04-16T15:40:57.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dcca3640 1 -- 172.21.15.71:0/466076631 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0034a20 con 0x7f18c4004d90 2024-04-16T15:40:57.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dff09640 1 -- 172.21.15.71:0/466076631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18c4004d90 msgr2=0x7f18c4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dff09640 1 --2- 172.21.15.71:0/466076631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18c4004d90 0x7f18c4005210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18d000a810 tx=0x7f18c4005830 comp rx=0 tx=0).stop 2024-04-16T15:40:57.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18de4a6640 1 -- 172.21.15.71:0/466076631 reap_dead start 2024-04-16T15:40:57.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dff09640 1 -- 172.21.15.71:0/466076631 shutdown_connections 2024-04-16T15:40:57.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dff09640 1 -- 172.21.15.71:0/466076631 >> 172.21.15.71:0/466076631 conn(0x7f18d80c5db0 msgr2=0x7f18d80c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:57.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dff09640 1 -- 172.21.15.71:0/466076631 shutdown_connections 2024-04-16T15:40:57.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.370+0000 7f18dff09640 1 -- 172.21.15.71:0/466076631 wait complete. 2024-04-16T15:40:57.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.371+0000 7f18dff09640 1 Processor -- start 2024-04-16T15:40:57.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18dff09640 1 -- start start 2024-04-16T15:40:57.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18dff09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4004d90 0x7f18d8151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18dff09640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d81573f0 0x7f18d81eed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18dff09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8150070 0x7f18d81ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18dff09640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18d8072450 con 0x7f18d8150070 2024-04-16T15:40:57.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18dff09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18d80722d0 con 0x7f18c4004d90 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18dff09640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f18d8156900 con 0x7f18d81573f0 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18de4a6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8150070 0x7f18d81ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18de4a6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8150070 0x7f18d81ef250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53052/0 (socket says 172.21.15.71:53052) 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18de4a6640 1 -- 172.21.15.71:0/239361597 learned_addr learned my addr 172.21.15.71:0/239361597 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.372+0000 7f18ddca5640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4004d90 0x7f18d8151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18dd4a4640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d81573f0 0x7f18d81eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18de4a6640 1 -- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d81573f0 msgr2=0x7f18d81eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18de4a6640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d81573f0 0x7f18d81eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18de4a6640 1 -- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4004d90 msgr2=0x7f18d8151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18de4a6640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4004d90 0x7f18d8151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18de4a6640 1 -- 172.21.15.71:0/239361597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d80ac990 con 0x7f18d8150070 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18dd4a4640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d81573f0 0x7f18d81eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18de4a6640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8150070 0x7f18d81ef250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f18d002cc20 tx=0x7f18d80c4200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18ca7fc640 1 -- 172.21.15.71:0/239361597 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4001370 con 0x7f18d8150070 2024-04-16T15:40:57.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18ca7fc640 1 -- 172.21.15.71:0/239361597 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d0044070 con 0x7f18d8150070 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d80acc70 con 0x7f18d8150070 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18ca7fc640 1 -- 172.21.15.71:0/239361597 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00b600 con 0x7f18d8150070 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.373+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d8151f90 con 0x7f18d8150070 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.374+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d8068be0 con 0x7f18d8150070 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.374+0000 7f18ca7fc640 1 -- 172.21.15.71:0/239361597 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18d0034bf0 con 0x7f18d8150070 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.375+0000 7f18ca7fc640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f18bc06c8f0 0x7f18bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.375+0000 7f18ca7fc640 1 -- 172.21.15.71:0/239361597 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f18d00ae050 con 0x7f18d8150070 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.375+0000 7f18ddca5640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f18bc06c8f0 0x7f18bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.375+0000 7f18ddca5640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f18bc06c8f0 0x7f18bc06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005550 tx=0x7f18d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.379+0000 7f18ca7fc640 1 -- 172.21.15.71:0/239361597 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18d007c2f0 con 0x7f18d8150070 2024-04-16T15:40:57.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.554+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18d80ac3f0 con 0x7f18d8150070 2024-04-16T15:40:57.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.554+0000 7f18ca7fc640 1 -- 172.21.15.71:0/239361597 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f18d007bbf0 con 0x7f18d8150070 2024-04-16T15:40:57.553 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f18bc06c8f0 msgr2=0x7f18bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f18bc06c8f0 0x7f18bc06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005550 tx=0x7f18d4009040 comp rx=0 tx=0).stop 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8150070 msgr2=0x7f18d81ef250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8150070 0x7f18d81ef250 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f18d002cc20 tx=0x7f18d80c4200 comp rx=0 tx=0).stop 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 shutdown_connections 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f18bc06c8f0 0x7f18bc06edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18d8150070 0x7f18d81ef250 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18d81573f0 0x7f18d81eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 --2- 172.21.15.71:0/239361597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c4004d90 0x7f18d8151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 >> 172.21.15.71:0/239361597 conn(0x7f18d80c5db0 msgr2=0x7f18d80aa890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 shutdown_connections 2024-04-16T15:40:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.556+0000 7f18dff09640 1 -- 172.21.15.71:0/239361597 wait complete. 2024-04-16T15:40:57.566 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:40:57.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.672+0000 7f31e900d640 1 Processor -- start 2024-04-16T15:40:57.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.672+0000 7f31e900d640 1 -- start start 2024-04-16T15:40:57.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e900d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f31e40c8ee0 0x7f31e40accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e900d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e41549a0 con 0x7f31e40c8130 2024-04-16T15:40:57.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e900d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e40c7650 con 0x7f31e40713a0 2024-04-16T15:40:57.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e900d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f31e40c7400 con 0x7f31e40c8ee0 2024-04-16T15:40:57.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31dbfff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f31e40c8130 0x7f31e40ac5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33510/0 (socket says 172.21.15.71:33510) 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31dbfff640 1 -- 172.21.15.71:0/4190995826 learned_addr learned my addr 172.21.15.71:0/4190995826 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153862253 0 0) 0x7f31e41549a0 con 0x7f31e40c8130 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d0003520 con 0x7f31e40c8130 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e359e640 1 --2- 172.21.15.71:0/4190995826 >> v2:172.21.15.122:3300/0 conn(0x7f31e40c8ee0 0x7f31e40accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1759221440 0 0) 0x7f31d0003520 con 0x7f31e40c8130 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 >> v2:172.21.15.122:3300/0 conn(0x7f31e40c8ee0 msgr2=0x7f31e40accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 --2- 172.21.15.71:0/4190995826 >> v2:172.21.15.122:3300/0 conn(0x7f31e40c8ee0 0x7f31e40accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 >> v1:172.21.15.122:6789/0 conn(0x7f31e40713a0 legacy=0x7f31e4155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40c7930 con 0x7f31e40c8130 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f31cc001670 con 0x7f31e40c8130 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 >> v1:172.21.15.71:6789/0 conn(0x7f31e40c8130 legacy=0x7f31e40ac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d00039b0 0x7f31d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31d0004800 0x7f31d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.673+0000 7f31e259c640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00051c0 0x7f31d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40c7400 con 0x7f31d00039b0 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31d000c160 con 0x7f31d0004800 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f31d000c360 con 0x7f31d00051c0 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e2d9d640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d00039b0 0x7f31d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31dbfff640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31d0004800 0x7f31d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e359e640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00051c0 0x7f31d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e2d9d640 1 -- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00051c0 msgr2=0x7f31d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e2d9d640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00051c0 0x7f31d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e2d9d640 1 -- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31d0004800 msgr2=0x7f31d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e2d9d640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31d0004800 0x7f31d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e2d9d640 1 -- 172.21.15.71:0/4190995826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d0003520 con 0x7f31d00039b0 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31e359e640 1 -- 172.21.15.71:0/4190995826 reap_dead start 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.674+0000 7f31dbfff640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31d0004800 0x7f31d0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e2d9d640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d00039b0 0x7f31d0003dd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31d4001750 tx=0x7f31d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3a0091571210f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4001370 con 0x7f31d00039b0 2024-04-16T15:40:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d40052d0 con 0x7f31d00039b0 2024-04-16T15:40:57.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e259c640 1 -- 172.21.15.71:0/4190995826 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4003830 con 0x7f31d00039b0 2024-04-16T15:40:57.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e900d640 1 -- 172.21.15.71:0/4190995826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d00039b0 msgr2=0x7f31d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e900d640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d00039b0 0x7f31d0003dd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31d4001750 tx=0x7f31d0005830 comp rx=0 tx=0).stop 2024-04-16T15:40:57.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e900d640 1 -- 172.21.15.71:0/4190995826 shutdown_connections 2024-04-16T15:40:57.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e900d640 1 --2- 172.21.15.71:0/4190995826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d00039b0 0x7f31d0003dd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e900d640 1 -- 172.21.15.71:0/4190995826 >> 172.21.15.71:0/4190995826 conn(0x7f31e40c96e0 msgr2=0x7f31e40c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:57.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e900d640 1 -- 172.21.15.71:0/4190995826 shutdown_connections 2024-04-16T15:40:57.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.675+0000 7f31e900d640 1 -- 172.21.15.71:0/4190995826 wait complete. 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.676+0000 7f31e900d640 1 Processor -- start 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31e900d640 1 -- start start 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31e900d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00039b0 0x7f31e4152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31e900d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e414a300 0x7f31e41497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31e900d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e40c63f0 0x7f31e4149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31e900d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4154590 con 0x7f31e414a300 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31e900d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40c7ca0 con 0x7f31e40c63f0 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31e900d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f31e4154710 con 0x7f31d00039b0 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e414a300 0x7f31e41497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e414a300 0x7f31e41497b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54778/0 (socket says 172.21.15.71:54778) 2024-04-16T15:40:57.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31dbfff640 1 -- 172.21.15.71:0/715293008 learned_addr learned my addr 172.21.15.71:0/715293008 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.677+0000 7f31e359e640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e40c63f0 0x7f31e4149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31e2d9d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00039b0 0x7f31e4152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31dbfff640 1 -- 172.21.15.71:0/715293008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00039b0 msgr2=0x7f31e4152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31dbfff640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00039b0 0x7f31e4152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31dbfff640 1 -- 172.21.15.71:0/715293008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e40c63f0 msgr2=0x7f31e4149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31dbfff640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e40c63f0 0x7f31e4149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31dbfff640 1 -- 172.21.15.71:0/715293008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e414fea0 con 0x7f31e414a300 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31dbfff640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e414a300 0x7f31e41497b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00e780 tx=0x7f31cc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31db7fe640 1 -- 172.21.15.71:0/715293008 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00c660 con 0x7f31e414a300 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31db7fe640 1 -- 172.21.15.71:0/715293008 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31cc018070 con 0x7f31e414a300 2024-04-16T15:40:57.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e414df30 con 0x7f31e414a300 2024-04-16T15:40:57.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31db7fe640 1 -- 172.21.15.71:0/715293008 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc00ca40 con 0x7f31e414a300 2024-04-16T15:40:57.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.678+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e414d8e0 con 0x7f31e414a300 2024-04-16T15:40:57.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.679+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e4068be0 con 0x7f31e414a300 2024-04-16T15:40:57.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.679+0000 7f31db7fe640 1 -- 172.21.15.71:0/715293008 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31cc00cbe0 con 0x7f31e414a300 2024-04-16T15:40:57.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.680+0000 7f31db7fe640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.680+0000 7f31db7fe640 1 -- 172.21.15.71:0/715293008 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f31cc08bbc0 con 0x7f31e414a300 2024-04-16T15:40:57.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.680+0000 7f31e2d9d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.680+0000 7f31e2d9d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f31c406c8f0 0x7f31c406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f31d4033000 tx=0x7f31d4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.684+0000 7f31db7fe640 1 -- 172.21.15.71:0/715293008 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31cc059d40 con 0x7f31e414a300 2024-04-16T15:40:57.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.862+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f31e41e9070 con 0x7f31e414a300 2024-04-16T15:40:57.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.862+0000 7f31db7fe640 1 -- 172.21.15.71:0/715293008 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f31cc059640 con 0x7f31e414a300 2024-04-16T15:40:57.861 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f31c406c8f0 msgr2=0x7f31c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f31c406c8f0 0x7f31c406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f31d4033000 tx=0x7f31d4009030 comp rx=0 tx=0).stop 2024-04-16T15:40:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e414a300 msgr2=0x7f31e41497b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e414a300 0x7f31e41497b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00e780 tx=0x7f31cc00ec40 comp rx=0 tx=0).stop 2024-04-16T15:40:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 shutdown_connections 2024-04-16T15:40:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f31c406c8f0 0x7f31c406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e40c63f0 0x7f31e4149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e414a300 0x7f31e41497b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 --2- 172.21.15.71:0/715293008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31d00039b0 0x7f31e4152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 >> 172.21.15.71:0/715293008 conn(0x7f31e40c96e0 msgr2=0x7f31e40ac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:57.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 shutdown_connections 2024-04-16T15:40:57.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.864+0000 7f31e900d640 1 -- 172.21.15.71:0/715293008 wait complete. 2024-04-16T15:40:57.874 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:57.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.978+0000 7f8f0d1db640 1 Processor -- start 2024-04-16T15:40:57.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.978+0000 7f8f0d1db640 1 -- start start 2024-04-16T15:40:57.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0d1db640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f080c4350 0x7f8f080c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0d1db640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f080c82d0 con 0x7f8f08153570 2024-04-16T15:40:57.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0d1db640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f08155d70 con 0x7f8f08159740 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0d1db640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0815a2e0 con 0x7f8f080c4350 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f06d9d640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f8f08153570 0x7f8f080c5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33520/0 (socket says 172.21.15.71:33520) 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f06d9d640 1 -- 172.21.15.71:0/364383770 learned_addr learned my addr 172.21.15.71:0/364383770 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2350652118 0 0) 0x7f8f080c82d0 con 0x7f8f08153570 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ef4003520 con 0x7f8f08153570 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8efe59c640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f080c4350 0x7f8f080c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 116000417 0 0) 0x7f8ef4003520 con 0x7f8f08153570 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f080c4350 msgr2=0x7f8f080c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f080c4350 0x7f8f080c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 >> v1:172.21.15.122:6789/0 conn(0x7f8f08159740 legacy=0x7f8f08159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f080c8d50 con 0x7f8f08153570 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f8ef8001670 con 0x7f8f08153570 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 >> v1:172.21.15.71:6789/0 conn(0x7f8f08153570 legacy=0x7f8f080c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8ef4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ef4004800 0x7f8ef4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f080c4350 0x7f8ef4005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0815a2e0 con 0x7f8ef40039e0 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef400bb80 con 0x7f8ef4004800 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef400bd80 con 0x7f8f080c4350 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f06d9d640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8ef4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.979+0000 7f8f0759e640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f080c4350 0x7f8ef4005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8efe59c640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ef4004800 0x7f8ef4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8f06d9d640 1 -- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f080c4350 msgr2=0x7f8ef4005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8f06d9d640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f080c4350 0x7f8ef4005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8f06d9d640 1 -- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ef4004800 msgr2=0x7f8ef4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:57.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8f06d9d640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ef4004800 0x7f8ef4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8f06d9d640 1 -- 172.21.15.71:0/364383770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef4003520 con 0x7f8ef40039e0 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8efe59c640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ef4004800 0x7f8ef4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8f06d9d640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8ef4003e00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c240 tx=0x7f8ef4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0d1427a11a151b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.980+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8007d10 con 0x7f8ef40039e0 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef8030070 con 0x7f8ef40039e0 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0659c640 1 -- 172.21.15.71:0/364383770 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef802cd80 con 0x7f8ef40039e0 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0d1db640 1 -- 172.21.15.71:0/364383770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 msgr2=0x7f8ef4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0d1db640 1 --2- 172.21.15.71:0/364383770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8ef4003e00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c240 tx=0x7f8ef4005830 comp rx=0 tx=0).stop 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0759e640 1 -- 172.21.15.71:0/364383770 reap_dead start 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0d1db640 1 -- 172.21.15.71:0/364383770 shutdown_connections 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0d1db640 1 -- 172.21.15.71:0/364383770 >> 172.21.15.71:0/364383770 conn(0x7f8f080c96e0 msgr2=0x7f8f080c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0d1db640 1 -- 172.21.15.71:0/364383770 shutdown_connections 2024-04-16T15:40:57.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.981+0000 7f8f0d1db640 1 -- 172.21.15.71:0/364383770 wait complete. 2024-04-16T15:40:57.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.982+0000 7f8f0d1db640 1 Processor -- start 2024-04-16T15:40:57.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f0d1db640 1 -- start start 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f0d1db640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8f08157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f0d1db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f081dfb90 0x7f8f0814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f0d1db640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f0814c5b0 0x7f8f081ebfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f0d1db640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f080c8ff0 con 0x7f8ef40039e0 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f0d1db640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f080c64b0 con 0x7f8f081dfb90 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f0d1db640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8f080c6630 con 0x7f8f0814c5b0 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f06d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8f08157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f06d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8f08157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54792/0 (socket says 172.21.15.71:54792) 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.983+0000 7f8f06d9d640 1 -- 172.21.15.71:0/762391867 learned_addr learned my addr 172.21.15.71:0/762391867 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f0759e640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f0814c5b0 0x7f8f081ebfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8efe59c640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f081dfb90 0x7f8f0814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f06d9d640 1 -- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f0814c5b0 msgr2=0x7f8f081ebfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f06d9d640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f0814c5b0 0x7f8f081ebfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f06d9d640 1 -- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f081dfb90 msgr2=0x7f8f0814c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f06d9d640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f081dfb90 0x7f8f0814c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f06d9d640 1 -- 172.21.15.71:0/762391867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0814cb80 con 0x7f8ef40039e0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8efe59c640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f081dfb90 0x7f8f0814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f0759e640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f0814c5b0 0x7f8f081ebfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f06d9d640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8f08157310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8001970 tx=0x7f8f080c5bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8efffff640 1 -- 172.21.15.71:0/762391867 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8045070 con 0x7f8ef40039e0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8efffff640 1 -- 172.21.15.71:0/762391867 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef80049f0 con 0x7f8ef40039e0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8efffff640 1 -- 172.21.15.71:0/762391867 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8007d10 con 0x7f8ef40039e0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0814e490 con 0x7f8ef40039e0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.984+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0814ea10 con 0x7f8ef40039e0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.985+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f08068d20 con 0x7f8ef40039e0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.985+0000 7f8efffff640 1 -- 172.21.15.71:0/762391867 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ef8007070 con 0x7f8ef40039e0 2024-04-16T15:40:57.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.986+0000 7f8efffff640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:57.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.986+0000 7f8efffff640 1 -- 172.21.15.71:0/762391867 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8ef80ace40 con 0x7f8ef40039e0 2024-04-16T15:40:57.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.986+0000 7f8efe59c640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:57.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.986+0000 7f8efe59c640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8ee806c8f0 0x7f8ee806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0006d40 tx=0x7f8ef0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:57.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:57.990+0000 7f8efffff640 1 -- 172.21.15.71:0/762391867 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ef807b0a0 con 0x7f8ef40039e0 2024-04-16T15:40:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.168+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f080c7c50 con 0x7f8ef40039e0 2024-04-16T15:40:58.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.169+0000 7f8efffff640 1 -- 172.21.15.71:0/762391867 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f8ef807a9a0 con 0x7f8ef40039e0 2024-04-16T15:40:58.168 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:58.168 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8ee806c8f0 msgr2=0x7f8ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8ee806c8f0 0x7f8ee806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0006d40 tx=0x7f8ef0008040 comp rx=0 tx=0).stop 2024-04-16T15:40:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 msgr2=0x7f8f08157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8f08157310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8001970 tx=0x7f8f080c5bd0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 shutdown_connections 2024-04-16T15:40:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f0814c5b0 0x7f8f081ebfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f081dfb90 0x7f8f0814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 --2- 172.21.15.71:0/762391867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef40039e0 0x7f8f08157310 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 >> 172.21.15.71:0/762391867 conn(0x7f8f080c96e0 msgr2=0x7f8f080c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:58.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 shutdown_connections 2024-04-16T15:40:58.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.171+0000 7f8f0d1db640 1 -- 172.21.15.71:0/762391867 wait complete. 2024-04-16T15:40:58.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:40:58.182 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.288+0000 7f776f7d4640 1 Processor -- start 2024-04-16T15:40:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.288+0000 7f776f7d4640 1 -- start start 2024-04-16T15:40:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776f7d4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77680735c0 0x7f77680c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776f7d4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7768152c90 0x7f7768153120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776f7d4640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77680c1340 con 0x7f7768159a40 2024-04-16T15:40:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776f7d4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f77681595b0 con 0x7f7768152c90 2024-04-16T15:40:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776f7d4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f77680c1620 con 0x7f77680735c0 2024-04-16T15:40:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776cd6f640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f7768159a40 0x7f7768155ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33522/0 (socket says 172.21.15.71:33522) 2024-04-16T15:40:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776cd6f640 1 -- 172.21.15.71:0/1086895710 learned_addr learned my addr 172.21.15.71:0/1086895710 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150722866 0 0) 0x7f77680c1340 con 0x7f7768159a40 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7754003520 con 0x7f7768159a40 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776dd71640 1 --2- 172.21.15.71:0/1086895710 >> v2:172.21.15.122:3300/0 conn(0x7f7768152c90 0x7f7768153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.289+0000 7f776d570640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77680735c0 0x7f77680c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3553244670 0 0) 0x7f7754003520 con 0x7f7768159a40 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77680735c0 msgr2=0x7f77680c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77680735c0 0x7f77680c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 >> v2:172.21.15.122:3300/0 conn(0x7f7768152c90 msgr2=0x7f7768153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 --2- 172.21.15.71:0/1086895710 >> v2:172.21.15.122:3300/0 conn(0x7f7768152c90 0x7f7768153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7768159240 con 0x7f7768159a40 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f7758001670 con 0x7f7768159a40 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 >> v1:172.21.15.71:6789/0 conn(0x7f7768159a40 legacy=0x7f7768155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77540039b0 0x7f7754003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77680735c0 0x7f7754004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7754004d40 0x7f77540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77681595b0 con 0x7f77540039b0 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77680c1620 con 0x7f7754004d40 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f775400ba40 con 0x7f77680735c0 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f776d570640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77540039b0 0x7f7754003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f776cd6f640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77680735c0 0x7f7754004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f776dd71640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7754004d40 0x7f77540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f776d570640 1 -- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77680735c0 msgr2=0x7f7754004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f776d570640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77680735c0 0x7f7754004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f776d570640 1 -- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7754004d40 msgr2=0x7f77540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f776d570640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7754004d40 0x7f77540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.290+0000 7f776d570640 1 -- 172.21.15.71:0/1086895710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7754003520 con 0x7f77540039b0 2024-04-16T15:40:58.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f776d570640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77540039b0 0x7f7754003dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7764009fc0 tx=0x7f7754005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e49bd2f9d3534f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764039070 con 0x7f77540039b0 2024-04-16T15:40:58.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776402cc20 con 0x7f77540039b0 2024-04-16T15:40:58.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f775ffff640 1 -- 172.21.15.71:0/1086895710 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764009c10 con 0x7f77540039b0 2024-04-16T15:40:58.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f776f7d4640 1 -- 172.21.15.71:0/1086895710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77540039b0 msgr2=0x7f7754003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f776f7d4640 1 --2- 172.21.15.71:0/1086895710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77540039b0 0x7f7754003dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7764009fc0 tx=0x7f7754005830 comp rx=0 tx=0).stop 2024-04-16T15:40:58.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f776dd71640 1 -- 172.21.15.71:0/1086895710 reap_dead start 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f776f7d4640 1 -- 172.21.15.71:0/1086895710 shutdown_connections 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.291+0000 7f776f7d4640 1 -- 172.21.15.71:0/1086895710 >> 172.21.15.71:0/1086895710 conn(0x7f776806b930 msgr2=0x7f776806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.292+0000 7f776f7d4640 1 -- 172.21.15.71:0/1086895710 shutdown_connections 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.292+0000 7f776f7d4640 1 -- 172.21.15.71:0/1086895710 wait complete. 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776f7d4640 1 Processor -- start 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776f7d4640 1 -- start start 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776f7d4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77540039b0 0x7f77681ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776f7d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77680c5900 0x7f77680ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776f7d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77680abe90 0x7f7768157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776f7d4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77680a7600 con 0x7f77680abe90 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776f7d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77680a7480 con 0x7f77680c5900 2024-04-16T15:40:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776f7d4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f77680739a0 con 0x7f77540039b0 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776dd71640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77680abe90 0x7f7768157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776dd71640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77680abe90 0x7f7768157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54816/0 (socket says 172.21.15.71:54816) 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.293+0000 7f776dd71640 1 -- 172.21.15.71:0/840405008 learned_addr learned my addr 172.21.15.71:0/840405008 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776cd6f640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77680c5900 0x7f77680ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776d570640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77540039b0 0x7f77681ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776dd71640 1 -- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77540039b0 msgr2=0x7f77681ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776dd71640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77540039b0 0x7f77681ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776dd71640 1 -- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77680c5900 msgr2=0x7f77680ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776dd71640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77680c5900 0x7f77680ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776dd71640 1 -- 172.21.15.71:0/840405008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77681ef560 con 0x7f77680abe90 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776d570640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77540039b0 0x7f77681ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776dd71640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77680abe90 0x7f7768157eb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7768062ee0 tx=0x7f7760011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f775dffb640 1 -- 172.21.15.71:0/840405008 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760004030 con 0x7f77680abe90 2024-04-16T15:40:58.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f775dffb640 1 -- 172.21.15.71:0/840405008 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776001b070 con 0x7f77680abe90 2024-04-16T15:40:58.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77680ad2b0 con 0x7f77680abe90 2024-04-16T15:40:58.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f775dffb640 1 -- 172.21.15.71:0/840405008 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77600043f0 con 0x7f77680abe90 2024-04-16T15:40:58.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.294+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77680ad850 con 0x7f77680abe90 2024-04-16T15:40:58.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.295+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77680ada50 con 0x7f77680abe90 2024-04-16T15:40:58.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.296+0000 7f775dffb640 1 -- 172.21.15.71:0/840405008 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7760012040 con 0x7f77680abe90 2024-04-16T15:40:58.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.296+0000 7f775dffb640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.296+0000 7f775dffb640 1 -- 172.21.15.71:0/840405008 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f776008bc50 con 0x7f77680abe90 2024-04-16T15:40:58.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.296+0000 7f776d570640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.297+0000 7f776d570640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f774c06c8f0 0x7f774c06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7764036000 tx=0x7f776402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.301+0000 7f775dffb640 1 -- 172.21.15.71:0/840405008 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77680ada50 con 0x7f77680abe90 2024-04-16T15:40:58.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.474+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7768068be0 con 0x7f77680abe90 2024-04-16T15:40:58.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.474+0000 7f775dffb640 1 -- 172.21.15.71:0/840405008 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f77681ef560 con 0x7f77680abe90 2024-04-16T15:40:58.473 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:40:58.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f774c06c8f0 msgr2=0x7f774c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f774c06c8f0 0x7f774c06edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7764036000 tx=0x7f776402e000 comp rx=0 tx=0).stop 2024-04-16T15:40:58.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77680abe90 msgr2=0x7f7768157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77680abe90 0x7f7768157eb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7768062ee0 tx=0x7f7760011cf0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 shutdown_connections 2024-04-16T15:40:58.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77680abe90 0x7f7768157eb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77680c5900 0x7f77680ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 --2- 172.21.15.71:0/840405008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77540039b0 0x7f77681ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 >> 172.21.15.71:0/840405008 conn(0x7f776806b930 msgr2=0x7f77680707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:58.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 shutdown_connections 2024-04-16T15:40:58.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.476+0000 7f776f7d4640 1 -- 172.21.15.71:0/840405008 wait complete. 2024-04-16T15:40:58.486 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:40:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.593+0000 7fcd91f38640 1 Processor -- start 2024-04-16T15:40:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.593+0000 7fcd91f38640 1 -- start start 2024-04-16T15:40:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd91f38640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcd8c150f40 0x7fcd8c0accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd91f38640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c0ab950 0x7fcd8c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd91f38640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd8c153ba0 con 0x7fcd8c0ad210 2024-04-16T15:40:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd91f38640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fcd8c153900 con 0x7fcd8c150f40 2024-04-16T15:40:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd91f38640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c0ac4d0 con 0x7fcd8c0ab950 2024-04-16T15:40:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd8b7fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcd8c150f40 0x7fcd8c0accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd8b7fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcd8c150f40 0x7fcd8c0accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54824/0 (socket says 172.21.15.71:54824) 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd8b7fe640 1 -- 172.21.15.71:0/847103781 learned_addr learned my addr 172.21.15.71:0/847103781 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.594+0000 7fcd8bfff640 1 --2- 172.21.15.71:0/847103781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c0ab950 0x7fcd8c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.595+0000 7fcd8b7fe640 1 -- 172.21.15.71:0/847103781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c0ab950 msgr2=0x7fcd8c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.595+0000 7fcd8b7fe640 1 --2- 172.21.15.71:0/847103781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c0ab950 0x7fcd8c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.595+0000 7fcd8b7fe640 1 -- 172.21.15.71:0/847103781 >> v1:172.21.15.122:6789/0 conn(0x7fcd8c0ad210 legacy=0x7fcd8c0aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.595+0000 7fcd8b7fe640 1 -- 172.21.15.71:0/847103781 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c0ac7b0 con 0x7fcd8c150f40 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.595+0000 7fcd8b7fe640 1 --2- 172.21.15.71:0/847103781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c150f40 0x7fcd8c0accd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009870 tx=0x7fcd7c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf9eb6943b8ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.595+0000 7fcd8a7fc640 1 -- 172.21.15.71:0/847103781 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c02f840 con 0x7fcd8c150f40 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd8a7fc640 1 -- 172.21.15.71:0/847103781 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c02f9e0 con 0x7fcd8c150f40 2024-04-16T15:40:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd8a7fc640 1 -- 172.21.15.71:0/847103781 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c02fdb0 con 0x7fcd8c150f40 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd91f38640 1 -- 172.21.15.71:0/847103781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c150f40 msgr2=0x7fcd8c0accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd91f38640 1 --2- 172.21.15.71:0/847103781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c150f40 0x7fcd8c0accd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009870 tx=0x7fcd7c02ba20 comp rx=0 tx=0).stop 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd91f38640 1 -- 172.21.15.71:0/847103781 shutdown_connections 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd91f38640 1 --2- 172.21.15.71:0/847103781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c0ab950 0x7fcd8c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd91f38640 1 --2- 172.21.15.71:0/847103781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c150f40 0x7fcd8c0accd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd91f38640 1 -- 172.21.15.71:0/847103781 >> 172.21.15.71:0/847103781 conn(0x7fcd8c0c3e10 msgr2=0x7fcd8c0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd91f38640 1 -- 172.21.15.71:0/847103781 shutdown_connections 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.596+0000 7fcd91f38640 1 -- 172.21.15.71:0/847103781 wait complete. 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.597+0000 7fcd91f38640 1 Processor -- start 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.597+0000 7fcd91f38640 1 -- start start 2024-04-16T15:40:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd91f38640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd8c0ab950 0x7fcd8c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd91f38640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c150f40 0x7fcd8c13e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd91f38640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c146340 0x7fcd8c1467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd91f38640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c0aac50 con 0x7fcd8c146340 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd91f38640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c0aaad0 con 0x7fcd8c0ab950 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd91f38640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c06fdd0 con 0x7fcd8c150f40 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c146340 0x7fcd8c1467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c146340 0x7fcd8c1467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54836/0 (socket says 172.21.15.71:54836) 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8bfff640 1 -- 172.21.15.71:0/2746200196 learned_addr learned my addr 172.21.15.71:0/2746200196 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8b7fe640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd8c0ab950 0x7fcd8c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8affd640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c150f40 0x7fcd8c13e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8bfff640 1 -- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c150f40 msgr2=0x7fcd8c13e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8bfff640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c150f40 0x7fcd8c13e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8bfff640 1 -- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd8c0ab950 msgr2=0x7fcd8c151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8bfff640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd8c0ab950 0x7fcd8c151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8bfff640 1 -- 172.21.15.71:0/2746200196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c0c5ad0 con 0x7fcd8c146340 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.598+0000 7fcd8affd640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c150f40 0x7fcd8c13e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.599+0000 7fcd8bfff640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c146340 0x7fcd8c1467f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400d600 tx=0x7fcd8400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.599+0000 7fcd73fff640 1 -- 172.21.15.71:0/2746200196 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd840125f0 con 0x7fcd8c146340 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.599+0000 7fcd73fff640 1 -- 172.21.15.71:0/2746200196 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd84019070 con 0x7fcd8c146340 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.599+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8c1ea250 con 0x7fcd8c146340 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.599+0000 7fcd73fff640 1 -- 172.21.15.71:0/2746200196 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd840129f0 con 0x7fcd8c146340 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.599+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8c151a70 con 0x7fcd8c146340 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.600+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8c068be0 con 0x7fcd8c146340 2024-04-16T15:40:58.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.600+0000 7fcd73fff640 1 -- 172.21.15.71:0/2746200196 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd84012b90 con 0x7fcd8c146340 2024-04-16T15:40:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.601+0000 7fcd73fff640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd7806c8f0 0x7fcd7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.601+0000 7fcd73fff640 1 -- 172.21.15.71:0/2746200196 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcd8408c240 con 0x7fcd8c146340 2024-04-16T15:40:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.601+0000 7fcd8b7fe640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd7806c8f0 0x7fcd7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.601+0000 7fcd8b7fe640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd7806c8f0 0x7fcd7806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c036000 tx=0x7fcd7c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.605+0000 7fcd73fff640 1 -- 172.21.15.71:0/2746200196 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd8405a340 con 0x7fcd8c146340 2024-04-16T15:40:58.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.778+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcd8c0c7040 con 0x7fcd8c146340 2024-04-16T15:40:58.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.779+0000 7fcd73fff640 1 -- 172.21.15.71:0/2746200196 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fcd84059c40 con 0x7fcd8c146340 2024-04-16T15:40:58.778 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd7806c8f0 msgr2=0x7fcd7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd7806c8f0 0x7fcd7806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c036000 tx=0x7fcd7c008030 comp rx=0 tx=0).stop 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c146340 msgr2=0x7fcd8c1467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c146340 0x7fcd8c1467f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400d600 tx=0x7fcd8400dac0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 shutdown_connections 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd7806c8f0 0x7fcd7806edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd8c146340 0x7fcd8c1467f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd8c150f40 0x7fcd8c13e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 --2- 172.21.15.71:0/2746200196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd8c0ab950 0x7fcd8c151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 >> 172.21.15.71:0/2746200196 conn(0x7fcd8c0c3e10 msgr2=0x7fcd8c14ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 shutdown_connections 2024-04-16T15:40:58.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.781+0000 7fcd91f38640 1 -- 172.21.15.71:0/2746200196 wait complete. 2024-04-16T15:40:58.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:40:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.899+0000 7fa8d3bd5640 1 Processor -- start 2024-04-16T15:40:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.899+0000 7fa8d3bd5640 1 -- start start 2024-04-16T15:40:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d3bd5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa8cc071080 0x7fa8cc0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d3bd5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc0acd60 0x7fa8cc0c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d3bd5640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8cc156220 con 0x7fa8cc0aa260 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d3bd5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fa8cc156500 con 0x7fa8cc071080 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d3bd5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa8cc1567e0 con 0x7fa8cc0acd60 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d1170640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fa8cc0aa260 0x7fa8cc0c3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33534/0 (socket says 172.21.15.71:33534) 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d1170640 1 -- 172.21.15.71:0/1320097520 learned_addr learned my addr 172.21.15.71:0/1320097520 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d1971640 1 --2- 172.21.15.71:0/1320097520 >> v2:172.21.15.122:3300/0 conn(0x7fa8cc071080 0x7fa8cc0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d2172640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc0acd60 0x7fa8cc0c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72350008 0 0) 0x7fa8cc156220 con 0x7fa8cc0aa260 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8b8003520 con 0x7fa8cc0aa260 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 700499819 0 0) 0x7fa8b8003520 con 0x7fa8cc0aa260 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc0acd60 msgr2=0x7fa8cc0c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc0acd60 0x7fa8cc0c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 >> v2:172.21.15.122:3300/0 conn(0x7fa8cc071080 msgr2=0x7fa8cc0a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 --2- 172.21.15.71:0/1320097520 >> v2:172.21.15.122:3300/0 conn(0x7fa8cc071080 0x7fa8cc0a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc156ac0 con 0x7fa8cc0aa260 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fa8bc001670 con 0x7fa8cc0aa260 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.900+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 >> v1:172.21.15.71:6789/0 conn(0x7fa8cc0aa260 legacy=0x7fa8cc0c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d096f640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b80039b0 0x7fa8b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d096f640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d096f640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc0acd60 0x7fa8b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc156500 con 0x7fa8b8004800 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1567e0 con 0x7fa8b80039b0 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b800b9c0 con 0x7fa8cc0acd60 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d1170640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d1971640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b80039b0 0x7fa8b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d2172640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc0acd60 0x7fa8b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d1170640 1 -- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc0acd60 msgr2=0x7fa8b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d1170640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc0acd60 0x7fa8b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d1170640 1 -- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b80039b0 msgr2=0x7fa8b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d1170640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b80039b0 0x7fa8b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.901+0000 7fa8d1170640 1 -- 172.21.15.71:0/1320097520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b8003520 con 0x7fa8b8004800 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d1170640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c290 tx=0x7fa8b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cffe1fd329da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc007d50 con 0x7fa8b8004800 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8bc030070 con 0x7fa8b8004800 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d096f640 1 -- 172.21.15.71:0/1320097520 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc02cd80 con 0x7fa8b8004800 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1320097520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8b8004800 msgr2=0x7fa8b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d3bd5640 1 --2- 172.21.15.71:0/1320097520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8b8004800 0x7fa8b8004c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c290 tx=0x7fa8b8005830 comp rx=0 tx=0).stop 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d2172640 1 -- 172.21.15.71:0/1320097520 reap_dead start 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1320097520 shutdown_connections 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1320097520 >> 172.21.15.71:0/1320097520 conn(0x7fa8cc0c8470 msgr2=0x7fa8cc0c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.902+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1320097520 shutdown_connections 2024-04-16T15:40:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.903+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1320097520 wait complete. 2024-04-16T15:40:58.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d3bd5640 1 Processor -- start 2024-04-16T15:40:58.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d3bd5640 1 -- start start 2024-04-16T15:40:58.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d3bd5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b8004800 0x7fa8cc1ec030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d3bd5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8cc06cd20 0x7fa8cc1ef190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d3bd5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc1ef6d0 0x7fa8cc1e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d3bd5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc0ac920 con 0x7fa8cc06cd20 2024-04-16T15:40:58.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d3bd5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc0ac7a0 con 0x7fa8b8004800 2024-04-16T15:40:58.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d3bd5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa8cc159f40 con 0x7fa8cc1ef6d0 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d1170640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8cc06cd20 0x7fa8cc1ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d1170640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8cc06cd20 0x7fa8cc1ef190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54850/0 (socket says 172.21.15.71:54850) 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.904+0000 7fa8d1170640 1 -- 172.21.15.71:0/1420879033 learned_addr learned my addr 172.21.15.71:0/1420879033 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d2172640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc1ef6d0 0x7fa8cc1e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d1971640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b8004800 0x7fa8cc1ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d1170640 1 -- 172.21.15.71:0/1420879033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc1ef6d0 msgr2=0x7fa8cc1e87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d1170640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc1ef6d0 0x7fa8cc1e87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d1170640 1 -- 172.21.15.71:0/1420879033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b8004800 msgr2=0x7fa8cc1ec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d1170640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b8004800 0x7fa8cc1ec030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d1170640 1 -- 172.21.15.71:0/1420879033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc158020 con 0x7fa8cc06cd20 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d1170640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8cc06cd20 0x7fa8cc1ef190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c380 tx=0x7fa8cc1563f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8c27fc640 1 -- 172.21.15.71:0/1420879033 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc045070 con 0x7fa8cc06cd20 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8c27fc640 1 -- 172.21.15.71:0/1420879033 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8bc000ed0 con 0x7fa8cc06cd20 2024-04-16T15:40:58.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc158850 con 0x7fa8cc06cd20 2024-04-16T15:40:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8c27fc640 1 -- 172.21.15.71:0/1420879033 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc001c80 con 0x7fa8cc06cd20 2024-04-16T15:40:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.905+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc158570 con 0x7fa8cc06cd20 2024-04-16T15:40:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.906+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc1efae0 con 0x7fa8cc06cd20 2024-04-16T15:40:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.906+0000 7fa8c27fc640 1 -- 172.21.15.71:0/1420879033 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8bc007070 con 0x7fa8cc06cd20 2024-04-16T15:40:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.907+0000 7fa8c27fc640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b006c8f0 0x7fa8b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.907+0000 7fa8c27fc640 1 -- 172.21.15.71:0/1420879033 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa8bc0acf10 con 0x7fa8cc06cd20 2024-04-16T15:40:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.907+0000 7fa8d1971640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b006c8f0 0x7fa8b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.907+0000 7fa8d1971640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b006c8f0 0x7fa8b006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009980 tx=0x7fa8c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:58.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:58.911+0000 7fa8c27fc640 1 -- 172.21.15.71:0/1420879033 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8bc07b170 con 0x7fa8cc06cd20 2024-04-16T15:40:59.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.088+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa8cc0ad970 con 0x7fa8cc06cd20 2024-04-16T15:40:59.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.088+0000 7fa8c27fc640 1 -- 172.21.15.71:0/1420879033 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fa8bc07aa70 con 0x7fa8cc06cd20 2024-04-16T15:40:59.087 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b006c8f0 msgr2=0x7fa8b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b006c8f0 0x7fa8b006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009980 tx=0x7fa8c8008040 comp rx=0 tx=0).stop 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8cc06cd20 msgr2=0x7fa8cc1ef190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8cc06cd20 0x7fa8cc1ef190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c380 tx=0x7fa8cc1563f0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 shutdown_connections 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa8b006c8f0 0x7fa8b006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa8cc1ef6d0 0x7fa8cc1e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8cc06cd20 0x7fa8cc1ef190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 --2- 172.21.15.71:0/1420879033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8b8004800 0x7fa8cc1ec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 >> 172.21.15.71:0/1420879033 conn(0x7fa8cc0c8470 msgr2=0x7fa8cc154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:59.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 shutdown_connections 2024-04-16T15:40:59.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.090+0000 7fa8d3bd5640 1 -- 172.21.15.71:0/1420879033 wait complete. 2024-04-16T15:40:59.100 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:40:59.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.207+0000 7f2cc657c640 1 Processor -- start 2024-04-16T15:40:59.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.207+0000 7f2cc657c640 1 -- start start 2024-04-16T15:40:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cc657c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2cc0155dc0 0x7f2cc00c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cc657c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc006fac0 0x7f2cc01561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cc657c640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cc00c27c0 con 0x7f2cc00ab290 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cc657c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc0073f20 con 0x7f2cc0155dc0 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cc657c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc0156ab0 con 0x7f2cc006fac0 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cc4b19640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f2cc00ab290 0x7f2cc01527d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33536/0 (socket says 172.21.15.71:33536) 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cc4b19640 1 -- 172.21.15.71:0/2627275591 learned_addr learned my addr 172.21.15.71:0/2627275591 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbf7fe640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc006fac0 0x7f2cc01561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbffff640 1 --2- 172.21.15.71:0/2627275591 >> v2:172.21.15.122:3300/0 conn(0x7f2cc0155dc0 0x7f2cc00c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295666810 0 0) 0x7f2cc00c27c0 con 0x7f2cc00ab290 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ca8003520 con 0x7f2cc00ab290 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2342651060 0 0) 0x7f2ca8003520 con 0x7f2cc00ab290 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc006fac0 msgr2=0x7f2cc01561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc006fac0 0x7f2cc01561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 >> v2:172.21.15.122:3300/0 conn(0x7f2cc0155dc0 msgr2=0x7f2cc00c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 --2- 172.21.15.71:0/2627275591 >> v2:172.21.15.122:3300/0 conn(0x7f2cc0155dc0 0x7f2cc00c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc00c2aa0 con 0x7f2cc00ab290 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f2cb80019f0 con 0x7f2cc00ab290 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbf7fe640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc006fac0 0x7f2cc01561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 >> v1:172.21.15.71:6789/0 conn(0x7f2cc00ab290 legacy=0x7f2cc01527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2ca8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ca8004800 0x7f2ca8004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca8005140 0x7f2ca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0156ab0 con 0x7f2ca80039b0 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80044d0 con 0x7f2ca8005140 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.208+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca800bf00 con 0x7f2ca8004800 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cbffff640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2ca8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cbf7fe640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ca8004800 0x7f2ca8004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cc4b19640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca8005140 0x7f2ca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cbffff640 1 -- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ca8004800 msgr2=0x7f2ca8004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cbffff640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ca8004800 0x7f2ca8004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cbffff640 1 -- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca8005140 msgr2=0x7f2ca800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cbffff640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca8005140 0x7f2ca800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cbffff640 1 -- 172.21.15.71:0/2627275591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca8003520 con 0x7f2ca80039b0 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.209+0000 7f2cc4b19640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca8005140 0x7f2ca800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cbffff640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2ca8003dd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009ac0 tx=0x7f2ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a006b7edb47f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0039070 con 0x7f2ca80039b0 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb002c710 con 0x7f2ca80039b0 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cbeffd640 1 -- 172.21.15.71:0/2627275591 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0009690 con 0x7f2ca80039b0 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cc657c640 1 -- 172.21.15.71:0/2627275591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 msgr2=0x7f2ca8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cc657c640 1 --2- 172.21.15.71:0/2627275591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2ca8003dd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009ac0 tx=0x7f2ca8005830 comp rx=0 tx=0).stop 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cc4b19640 1 -- 172.21.15.71:0/2627275591 reap_dead start 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cc657c640 1 -- 172.21.15.71:0/2627275591 shutdown_connections 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cc657c640 1 -- 172.21.15.71:0/2627275591 >> 172.21.15.71:0/2627275591 conn(0x7f2cc00c3e10 msgr2=0x7f2cc00c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:59.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cc657c640 1 -- 172.21.15.71:0/2627275591 shutdown_connections 2024-04-16T15:40:59.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.210+0000 7f2cc657c640 1 -- 172.21.15.71:0/2627275591 wait complete. 2024-04-16T15:40:59.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.211+0000 7f2cc657c640 1 Processor -- start 2024-04-16T15:40:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cc657c640 1 -- start start 2024-04-16T15:40:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cc657c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2cc0072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cc657c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc0155960 0x7f2cc01ef610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cc657c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00c4e70 0x7f2cc01efb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cc657c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc006bd70 con 0x7f2ca80039b0 2024-04-16T15:40:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cc657c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc00c18b0 con 0x7f2cc00c4e70 2024-04-16T15:40:59.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cc657c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc006bef0 con 0x7f2cc0155960 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cbffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2cc0072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cbffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2cc0072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54874/0 (socket says 172.21.15.71:54874) 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cbffff640 1 -- 172.21.15.71:0/308086788 learned_addr learned my addr 172.21.15.71:0/308086788 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cc4b19640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00c4e70 0x7f2cc01efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.212+0000 7f2cbf7fe640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc0155960 0x7f2cc01ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbffff640 1 -- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc0155960 msgr2=0x7f2cc01ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbffff640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc0155960 0x7f2cc01ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbffff640 1 -- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00c4e70 msgr2=0x7f2cc01efb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbffff640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00c4e70 0x7f2cc01efb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbffff640 1 -- 172.21.15.71:0/308086788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc0139c60 con 0x7f2ca80039b0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cc4b19640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00c4e70 0x7f2cc01efb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbf7fe640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc0155960 0x7f2cc01ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbffff640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2cc0072c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0000c00 tx=0x7f2cb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbcff9640 1 -- 172.21.15.71:0/308086788 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0039050 con 0x7f2ca80039b0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbcff9640 1 -- 172.21.15.71:0/308086788 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb0045070 con 0x7f2ca80039b0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01552a0 con 0x7f2ca80039b0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cbcff9640 1 -- 172.21.15.71:0/308086788 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00044e0 con 0x7f2ca80039b0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.213+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc0155540 con 0x7f2ca80039b0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.214+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc006d9e0 con 0x7f2ca80039b0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.214+0000 7f2cbcff9640 1 -- 172.21.15.71:0/308086788 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cb0003a50 con 0x7f2ca80039b0 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.215+0000 7f2cbcff9640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.215+0000 7f2cbcff9640 1 -- 172.21.15.71:0/308086788 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2cb00ad930 con 0x7f2ca80039b0 2024-04-16T15:40:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.215+0000 7f2cbf7fe640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.215+0000 7f2cbf7fe640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2cac06c8f0 0x7f2cac06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0155d60 tx=0x7f2cb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.219+0000 7f2cbcff9640 1 -- 172.21.15.71:0/308086788 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2cc006d9e0 con 0x7f2ca80039b0 2024-04-16T15:40:59.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.394+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2cc0068be0 con 0x7f2ca80039b0 2024-04-16T15:40:59.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.394+0000 7f2cbcff9640 1 -- 172.21.15.71:0/308086788 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f2cb004e020 con 0x7f2ca80039b0 2024-04-16T15:40:59.393 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:40:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2cac06c8f0 msgr2=0x7f2cac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2cac06c8f0 0x7f2cac06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0155d60 tx=0x7f2cb4008040 comp rx=0 tx=0).stop 2024-04-16T15:40:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 msgr2=0x7f2cc0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2cc0072c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0000c00 tx=0x7f2cb0004730 comp rx=0 tx=0).stop 2024-04-16T15:40:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 shutdown_connections 2024-04-16T15:40:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2cac06c8f0 0x7f2cac06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00c4e70 0x7f2cc01efb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cc0155960 0x7f2cc01ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 --2- 172.21.15.71:0/308086788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ca80039b0 0x7f2cc0072c80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 >> 172.21.15.71:0/308086788 conn(0x7f2cc00c3e10 msgr2=0x7f2cc00c1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 shutdown_connections 2024-04-16T15:40:59.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.396+0000 7f2cc657c640 1 -- 172.21.15.71:0/308086788 wait complete. 2024-04-16T15:40:59.406 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:40:59.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.510+0000 7f7f4b9c8640 1 Processor -- start 2024-04-16T15:40:59.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.511+0000 7f7f4b9c8640 1 -- start start 2024-04-16T15:40:59.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.511+0000 7f7f4b9c8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7f44155dc0 0x7f7f440c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.511+0000 7f7f4b9c8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 0x7f7f4415d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.511+0000 7f7f4b9c8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f440c27c0 con 0x7f7f4406fac0 2024-04-16T15:40:59.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.511+0000 7f7f4b9c8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7f44073f20 con 0x7f7f44155dc0 2024-04-16T15:40:59.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.511+0000 7f7f4b9c8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7f44156ab0 con 0x7f7f440ab290 2024-04-16T15:40:59.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f48f63640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f7f4406fac0 0x7f7f441561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33538/0 (socket says 172.21.15.71:33538) 2024-04-16T15:40:59.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f48f63640 1 -- 172.21.15.71:0/3819489425 learned_addr learned my addr 172.21.15.71:0/3819489425 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1468286693 0 0) 0x7f7f440c27c0 con 0x7f7f4406fac0 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f30003520 con 0x7f7f4406fac0 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49764640 1 --2- 172.21.15.71:0/3819489425 >> v2:172.21.15.122:3300/0 conn(0x7f7f44155dc0 0x7f7f440c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49f65640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 0x7f7f4415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1166139787 0 0) 0x7f7f30003520 con 0x7f7f4406fac0 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 msgr2=0x7f7f4415d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 0x7f7f4415d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 >> v2:172.21.15.122:3300/0 conn(0x7f7f44155dc0 msgr2=0x7f7f440c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 --2- 172.21.15.71:0/3819489425 >> v2:172.21.15.122:3300/0 conn(0x7f7f44155dc0 0x7f7f440c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f440c2aa0 con 0x7f7f4406fac0 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f7f40001670 con 0x7f7f4406fac0 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49f65640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 0x7f7f4415d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 >> v1:172.21.15.71:6789/0 conn(0x7f7f4406fac0 legacy=0x7f7f441561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 0x7f7f300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f30003fa0 0x7f7f30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44156ab0 con 0x7f7f30004d40 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000ba20 con 0x7f7f30003fa0 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3000bc20 con 0x7f7f440ab290 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49f65640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f48f63640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f30003fa0 0x7f7f30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49764640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 0x7f7f300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49f65640 1 -- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 msgr2=0x7f7f300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49f65640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f440ab290 0x7f7f300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49f65640 1 -- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f30003fa0 msgr2=0x7f7f30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49f65640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f30003fa0 0x7f7f30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.512+0000 7f7f49f65640 1 -- 172.21.15.71:0/3819489425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f30003520 con 0x7f7f30004d40 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.513+0000 7f7f49f65640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f300051c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0073e0 tx=0x7f7f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19150da5064440e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.513+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c039070 con 0x7f7f30004d40 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.513+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c02cd30 con 0x7f7f30004d40 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.513+0000 7f7f3bfff640 1 -- 172.21.15.71:0/3819489425 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c009c80 con 0x7f7f30004d40 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.514+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/3819489425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 msgr2=0x7f7f300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.514+0000 7f7f4b9c8640 1 --2- 172.21.15.71:0/3819489425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f300051c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0073e0 tx=0x7f7f30005830 comp rx=0 tx=0).stop 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.514+0000 7f7f49f65640 1 -- 172.21.15.71:0/3819489425 reap_dead start 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.514+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/3819489425 shutdown_connections 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.514+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/3819489425 >> 172.21.15.71:0/3819489425 conn(0x7f7f440c3e10 msgr2=0x7f7f440c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.514+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/3819489425 shutdown_connections 2024-04-16T15:40:59.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.514+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/3819489425 wait complete. 2024-04-16T15:40:59.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.515+0000 7f7f4b9c8640 1 Processor -- start 2024-04-16T15:40:59.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.515+0000 7f7f4b9c8640 1 -- start start 2024-04-16T15:40:59.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f4b9c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f4406c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f4b9c8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f441dbdc0 0x7f7f44144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f4b9c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f44145390 0x7f7f4413b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f4b9c8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4406bd70 con 0x7f7f30004d40 2024-04-16T15:40:59.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f4b9c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f440c18b0 con 0x7f7f44145390 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f4b9c8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4406bef0 con 0x7f7f441dbdc0 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f4406c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f4406c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54886/0 (socket says 172.21.15.71:54886) 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 -- 172.21.15.71:0/2857938323 learned_addr learned my addr 172.21.15.71:0/2857938323 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49f65640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f44145390 0x7f7f4413b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f48f63640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f441dbdc0 0x7f7f44144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 -- 172.21.15.71:0/2857938323 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f441dbdc0 msgr2=0x7f7f44144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f441dbdc0 0x7f7f44144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 -- 172.21.15.71:0/2857938323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f44145390 msgr2=0x7f7f4413b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f44145390 0x7f7f4413b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 -- 172.21.15.71:0/2857938323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f441dc880 con 0x7f7f30004d40 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f49764640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f4406c430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400b7f0 tx=0x7f7f3400bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f39ffb640 1 -- 172.21.15.71:0/2857938323 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34018070 con 0x7f7f30004d40 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f39ffb640 1 -- 172.21.15.71:0/2857938323 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f34010830 con 0x7f7f30004d40 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f39ffb640 1 -- 172.21.15.71:0/2857938323 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34010c00 con 0x7f7f30004d40 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.516+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f441466a0 con 0x7f7f30004d40 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.517+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4406c970 con 0x7f7f30004d40 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.518+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f44146490 con 0x7f7f30004d40 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.518+0000 7f7f39ffb640 1 -- 172.21.15.71:0/2857938323 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f3401b070 con 0x7f7f30004d40 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.518+0000 7f7f39ffb640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7f2806c920 0x7f7f2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.519+0000 7f7f39ffb640 1 -- 172.21.15.71:0/2857938323 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7f3408c170 con 0x7f7f30004d40 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.519+0000 7f7f48f63640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7f2806c920 0x7f7f2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.519+0000 7f7f48f63640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7f2806c920 0x7f7f2806ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7f4413fd60 tx=0x7f7f400078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.523+0000 7f7f39ffb640 1 -- 172.21.15.71:0/2857938323 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f34059ca0 con 0x7f7f30004d40 2024-04-16T15:40:59.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.699+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f4406bb50 con 0x7f7f30004d40 2024-04-16T15:40:59.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.700+0000 7f7f39ffb640 1 -- 172.21.15.71:0/2857938323 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f7f3405dfa0 con 0x7f7f30004d40 2024-04-16T15:40:59.699 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:40:59.699 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:40:59.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7f2806c920 msgr2=0x7f7f2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7f2806c920 0x7f7f2806ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7f4413fd60 tx=0x7f7f400078f0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 msgr2=0x7f7f4406c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f4406c430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400b7f0 tx=0x7f7f3400bbf0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 shutdown_connections 2024-04-16T15:40:59.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7f2806c920 0x7f7f2806ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f44145390 0x7f7f4413b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f441dbdc0 0x7f7f44144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 --2- 172.21.15.71:0/2857938323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30004d40 0x7f7f4406c430 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 >> 172.21.15.71:0/2857938323 conn(0x7f7f440c3e10 msgr2=0x7f7f44140350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:59.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 shutdown_connections 2024-04-16T15:40:59.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.702+0000 7f7f4b9c8640 1 -- 172.21.15.71:0/2857938323 wait complete. 2024-04-16T15:40:59.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:40:59.713 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:40:59.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.820+0000 7f37eb4c1640 1 Processor -- start 2024-04-16T15:40:59.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.820+0000 7f37eb4c1640 1 -- start start 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37eb4c1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4159f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37eb4c1640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f37e40c3710 0x7f37e40c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37eb4c1640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e40acfb0 con 0x7f37e40ac5c0 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37eb4c1640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f37e4156220 con 0x7f37e40c3710 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37eb4c1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37e4156500 con 0x7f37e4070ff0 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37e925d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37e9a5e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f37e40c3710 0x7f37e40c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37e925d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:55264/0 (socket says 172.21.15.71:55264) 2024-04-16T15:40:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37e8a5c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f37e40ac5c0 0x7f37e40a9d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:54202/0 (socket says 172.21.15.71:54202) 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.821+0000 7f37e925d640 1 -- 172.21.15.71:0/2614202546 learned_addr learned my addr 172.21.15.71:0/2614202546 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.822+0000 7f37e9a5e640 1 -- 172.21.15.71:0/2614202546 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 msgr2=0x7f37e4159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.822+0000 7f37e9a5e640 1 --2- 172.21.15.71:0/2614202546 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.822+0000 7f37e9a5e640 1 -- 172.21.15.71:0/2614202546 >> v1:172.21.15.122:6789/0 conn(0x7f37e40ac5c0 legacy=0x7f37e40a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.822+0000 7f37e9a5e640 1 -- 172.21.15.71:0/2614202546 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e41567e0 con 0x7f37e40c3710 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.822+0000 7f37e9a5e640 1 --2- 172.21.15.71:0/2614202546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 0x7f37e40c3b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097a0 tx=0x7f37d402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be046fcc3280c6b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37e3fff640 1 -- 172.21.15.71:0/2614202546 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4035020 con 0x7f37e40c3710 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37e3fff640 1 -- 172.21.15.71:0/2614202546 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d4031830 con 0x7f37e40c3710 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37e3fff640 1 -- 172.21.15.71:0/2614202546 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4031c00 con 0x7f37e40c3710 2024-04-16T15:40:59.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37eb4c1640 1 -- 172.21.15.71:0/2614202546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 msgr2=0x7f37e40c3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/2614202546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 0x7f37e40c3b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097a0 tx=0x7f37d402b8d0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37eb4c1640 1 -- 172.21.15.71:0/2614202546 shutdown_connections 2024-04-16T15:40:59.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/2614202546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 0x7f37e40c3b20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/2614202546 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37eb4c1640 1 -- 172.21.15.71:0/2614202546 >> 172.21.15.71:0/2614202546 conn(0x7f37e40c8470 msgr2=0x7f37e40c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:40:59.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37eb4c1640 1 -- 172.21.15.71:0/2614202546 shutdown_connections 2024-04-16T15:40:59.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.823+0000 7f37eb4c1640 1 -- 172.21.15.71:0/2614202546 wait complete. 2024-04-16T15:40:59.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.824+0000 7f37eb4c1640 1 Processor -- start 2024-04-16T15:40:59.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.824+0000 7f37eb4c1640 1 -- start start 2024-04-16T15:40:59.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37eb4c1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37eb4c1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e40ac5c0 0x7f37e4151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37eb4c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 0x7f37e41ef410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37eb4c1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37e40aa5b0 con 0x7f37e40ac5c0 2024-04-16T15:40:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37eb4c1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37e40aa430 con 0x7f37e40c3710 2024-04-16T15:40:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37eb4c1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37e40713d0 con 0x7f37e4070ff0 2024-04-16T15:40:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e8a5c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e40ac5c0 0x7f37e4151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e8a5c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e40ac5c0 0x7f37e4151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54896/0 (socket says 172.21.15.71:54896) 2024-04-16T15:40:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e8a5c640 1 -- 172.21.15.71:0/3167391628 learned_addr learned my addr 172.21.15.71:0/3167391628 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:40:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e925d640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e9a5e640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 0x7f37e41ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e8a5c640 1 -- 172.21.15.71:0/3167391628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 msgr2=0x7f37e4154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e8a5c640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e8a5c640 1 -- 172.21.15.71:0/3167391628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 msgr2=0x7f37e41ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:40:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e8a5c640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 0x7f37e41ef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:40:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.825+0000 7f37e8a5c640 1 -- 172.21.15.71:0/3167391628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e4153cb0 con 0x7f37e40ac5c0 2024-04-16T15:40:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.826+0000 7f37e8a5c640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e40ac5c0 0x7f37e4151550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c7c0 tx=0x7f37d800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.826+0000 7f37e1ffb640 1 -- 172.21.15.71:0/3167391628 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8017070 con 0x7f37e40ac5c0 2024-04-16T15:40:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.826+0000 7f37e1ffb640 1 -- 172.21.15.71:0/3167391628 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d800a640 con 0x7f37e40ac5c0 2024-04-16T15:40:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.826+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e4153f90 con 0x7f37e40ac5c0 2024-04-16T15:40:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.826+0000 7f37e1ffb640 1 -- 172.21.15.71:0/3167391628 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d800a9f0 con 0x7f37e40ac5c0 2024-04-16T15:40:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.826+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e41eeef0 con 0x7f37e40ac5c0 2024-04-16T15:40:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.827+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e41ef0f0 con 0x7f37e40ac5c0 2024-04-16T15:40:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.827+0000 7f37e1ffb640 1 -- 172.21.15.71:0/3167391628 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37d801a070 con 0x7f37e40ac5c0 2024-04-16T15:40:59.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.828+0000 7f37e1ffb640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:40:59.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.828+0000 7f37e925d640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:40:59.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.828+0000 7f37e1ffb640 1 -- 172.21.15.71:0/3167391628 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f37d808b390 con 0x7f37e40ac5c0 2024-04-16T15:40:59.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.828+0000 7f37e925d640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37d006c8f0 0x7f37d006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f37e40c7140 tx=0x7f37dc009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:40:59.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:40:59.832+0000 7f37e1ffb640 1 -- 172.21.15.71:0/3167391628 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37e41ef0f0 con 0x7f37e40ac5c0 2024-04-16T15:41:00.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.008+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37e4068be0 con 0x7f37e40ac5c0 2024-04-16T15:41:00.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.008+0000 7f37e1ffb640 1 -- 172.21.15.71:0/3167391628 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f37d8058ec0 con 0x7f37e40ac5c0 2024-04-16T15:41:00.007 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:41:00.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37d006c8f0 msgr2=0x7f37d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37d006c8f0 0x7f37d006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f37e40c7140 tx=0x7f37dc009240 comp rx=0 tx=0).stop 2024-04-16T15:41:00.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e40ac5c0 msgr2=0x7f37e4151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e40ac5c0 0x7f37e4151550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c7c0 tx=0x7f37d800cc80 comp rx=0 tx=0).stop 2024-04-16T15:41:00.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 shutdown_connections 2024-04-16T15:41:00.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f37d006c8f0 0x7f37d006edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e40c3710 0x7f37e41ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37e40ac5c0 0x7f37e4151550 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 --2- 172.21.15.71:0/3167391628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37e4070ff0 0x7f37e4154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 >> 172.21.15.71:0/3167391628 conn(0x7f37e40c8470 msgr2=0x7f37e40c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:00.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 shutdown_connections 2024-04-16T15:41:00.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.010+0000 7f37eb4c1640 1 -- 172.21.15.71:0/3167391628 wait complete. 2024-04-16T15:41:00.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:41:00.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.129+0000 7fc913d00640 1 Processor -- start 2024-04-16T15:41:00.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.130+0000 7fc913d00640 1 -- start start 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.130+0000 7fc913d00640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc90c06fac0 0x7fc90c0c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.130+0000 7fc913d00640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc90c0ab290 0x7fc90c15d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.130+0000 7fc913d00640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c0c27c0 con 0x7fc90c155dc0 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.130+0000 7fc913d00640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c073f20 con 0x7fc90c0ab290 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.130+0000 7fc913d00640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c156ab0 con 0x7fc90c06fac0 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc911a9c640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fc90c155dc0 0x7fc90c1561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33548/0 (socket says 172.21.15.71:33548) 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc911a9c640 1 -- 172.21.15.71:0/1499395006 learned_addr learned my addr 172.21.15.71:0/1499395006 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc91229d640 1 --2- 172.21.15.71:0/1499395006 >> v2:172.21.15.122:3300/0 conn(0x7fc90c0ab290 0x7fc90c15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc91129b640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc90c06fac0 0x7fc90c0c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4185043786 0 0) 0x7fc90c0c27c0 con 0x7fc90c155dc0 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8f8003520 con 0x7fc90c155dc0 2024-04-16T15:41:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1151973051 0 0) 0x7fc8f8003520 con 0x7fc90c155dc0 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc90c06fac0 msgr2=0x7fc90c0c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc90c06fac0 0x7fc90c0c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 >> v2:172.21.15.122:3300/0 conn(0x7fc90c0ab290 msgr2=0x7fc90c15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 --2- 172.21.15.71:0/1499395006 >> v2:172.21.15.122:3300/0 conn(0x7fc90c0ab290 0x7fc90c15d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0c2aa0 con 0x7fc90c155dc0 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fc908001670 con 0x7fc90c155dc0 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc91229d640 1 --2- 172.21.15.71:0/1499395006 >> v2:172.21.15.122:3300/0 conn(0x7fc90c0ab290 0x7fc90c15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 >> v1:172.21.15.71:6789/0 conn(0x7fc90c155dc0 legacy=0x7fc90c1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f80039b0 0x7fc8f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8f8004800 0x7fc8f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80051c0 0x7fc8f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c156ab0 con 0x7fc8f80039b0 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f800bf00 con 0x7fc8f8004800 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f800c080 con 0x7fc8f80051c0 2024-04-16T15:41:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc91129b640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc90c06fac0 0x7fc90c0c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc911a9c640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f80039b0 0x7fc8f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.131+0000 7fc91229d640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80051c0 0x7fc8f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc91129b640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8f8004800 0x7fc8f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc911a9c640 1 -- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80051c0 msgr2=0x7fc8f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc911a9c640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80051c0 0x7fc8f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc911a9c640 1 -- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8f8004800 msgr2=0x7fc8f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc911a9c640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8f8004800 0x7fc8f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc911a9c640 1 -- 172.21.15.71:0/1499395006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f8003520 con 0x7fc8f80039b0 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc911a9c640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f80039b0 0x7fc8f8003dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c2c0 tx=0x7fc8f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e37329d9adbecbb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908007d50 con 0x7fc8f80039b0 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc908030070 con 0x7fc8f80039b0 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.132+0000 7fc910a9a640 1 -- 172.21.15.71:0/1499395006 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90802cd60 con 0x7fc8f80039b0 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.133+0000 7fc913d00640 1 -- 172.21.15.71:0/1499395006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f80039b0 msgr2=0x7fc8f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.133+0000 7fc913d00640 1 --2- 172.21.15.71:0/1499395006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8f80039b0 0x7fc8f8003dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc90800c2c0 tx=0x7fc8f8005830 comp rx=0 tx=0).stop 2024-04-16T15:41:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.133+0000 7fc91229d640 1 -- 172.21.15.71:0/1499395006 reap_dead start 2024-04-16T15:41:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.133+0000 7fc913d00640 1 -- 172.21.15.71:0/1499395006 shutdown_connections 2024-04-16T15:41:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.133+0000 7fc913d00640 1 -- 172.21.15.71:0/1499395006 >> 172.21.15.71:0/1499395006 conn(0x7fc90c0c3e10 msgr2=0x7fc90c0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.133+0000 7fc913d00640 1 -- 172.21.15.71:0/1499395006 shutdown_connections 2024-04-16T15:41:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.133+0000 7fc913d00640 1 -- 172.21.15.71:0/1499395006 wait complete. 2024-04-16T15:41:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.134+0000 7fc913d00640 1 Processor -- start 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.134+0000 7fc913d00640 1 -- start start 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc913d00640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80039b0 0x7fc90c06c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc913d00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc90c0c4990 0x7fc90c1efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc913d00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c13f680 0x7fc90c13fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc913d00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c06bd70 con 0x7fc90c13f680 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc913d00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c0c18b0 con 0x7fc90c0c4990 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc913d00640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c06bef0 con 0x7fc8f80039b0 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c13f680 0x7fc90c13fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c13f680 0x7fc90c13fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54902/0 (socket says 172.21.15.71:54902) 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 -- 172.21.15.71:0/59655529 learned_addr learned my addr 172.21.15.71:0/59655529 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc911a9c640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80039b0 0x7fc90c06c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91129b640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc90c0c4990 0x7fc90c1efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 -- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80039b0 msgr2=0x7fc90c06c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80039b0 0x7fc90c06c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 -- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc90c0c4990 msgr2=0x7fc90c1efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc90c0c4990 0x7fc90c1efac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 -- 172.21.15.71:0/59655529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0ac1e0 con 0x7fc90c13f680 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc911a9c640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80039b0 0x7fc90c06c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91129b640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc90c0c4990 0x7fc90c1efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.135+0000 7fc91229d640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c13f680 0x7fc90c13fb10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0c3de0 tx=0x7fc904009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.136+0000 7fc9027fc640 1 -- 172.21.15.71:0/59655529 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904011e00 con 0x7fc90c13f680 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.136+0000 7fc9027fc640 1 -- 172.21.15.71:0/59655529 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90401b070 con 0x7fc90c13f680 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.136+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c0ac4c0 con 0x7fc90c13f680 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.136+0000 7fc9027fc640 1 -- 172.21.15.71:0/59655529 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904003990 con 0x7fc90c13f680 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.136+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c1ef5a0 con 0x7fc90c13f680 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.137+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c068be0 con 0x7fc90c13f680 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.137+0000 7fc9027fc640 1 -- 172.21.15.71:0/59655529 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc904005d10 con 0x7fc90c13f680 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.138+0000 7fc9027fc640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc8f006c8f0 0x7fc8f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.138+0000 7fc911a9c640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc8f006c8f0 0x7fc8f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.138+0000 7fc9027fc640 1 -- 172.21.15.71:0/59655529 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc90408c480 con 0x7fc90c13f680 2024-04-16T15:41:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.138+0000 7fc911a9c640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc8f006c8f0 0x7fc8f006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc908034000 tx=0x7fc908039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.142+0000 7fc9027fc640 1 -- 172.21.15.71:0/59655529 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc904025020 con 0x7fc90c13f680 2024-04-16T15:41:00.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.316+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc90c06f650 con 0x7fc90c13f680 2024-04-16T15:41:00.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.317+0000 7fc9027fc640 1 -- 172.21.15.71:0/59655529 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fc904014350 con 0x7fc90c13f680 2024-04-16T15:41:00.316 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc8f006c8f0 msgr2=0x7fc8f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc8f006c8f0 0x7fc8f006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc908034000 tx=0x7fc908039040 comp rx=0 tx=0).stop 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c13f680 msgr2=0x7fc90c13fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c13f680 0x7fc90c13fb10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0c3de0 tx=0x7fc904009f90 comp rx=0 tx=0).stop 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 shutdown_connections 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc8f006c8f0 0x7fc8f006edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c13f680 0x7fc90c13fb10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc90c0c4990 0x7fc90c1efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 --2- 172.21.15.71:0/59655529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc8f80039b0 0x7fc90c06c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 >> 172.21.15.71:0/59655529 conn(0x7fc90c0c3e10 msgr2=0x7fc90c0c1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 shutdown_connections 2024-04-16T15:41:00.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.319+0000 7fc913d00640 1 -- 172.21.15.71:0/59655529 wait complete. 2024-04-16T15:41:00.328 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:41:00.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.435+0000 7fb07e2d9640 1 Processor -- start 2024-04-16T15:41:00.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.435+0000 7fb07e2d9640 1 -- start start 2024-04-16T15:41:00.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb07e2d9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb0780c32a0 0x7fb078155e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb07e2d9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c47b0 0x7fb0781563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb07e2d9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb07806cdb0 con 0x7fb0781494e0 2024-04-16T15:41:00.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb07e2d9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb07806cb60 con 0x7fb0780c32a0 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb07e2d9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb07806d090 con 0x7fb0780c47b0 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb07c876640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c47b0 0x7fb0781563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb0777fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb0780c32a0 0x7fb078155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb07c876640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c47b0 0x7fb0781563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:55296/0 (socket says 172.21.15.71:55296) 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb0777fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb0780c32a0 0x7fb078155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:54034/0 (socket says 172.21.15.71:54034) 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.436+0000 7fb07c876640 1 -- 172.21.15.71:0/3827828067 learned_addr learned my addr 172.21.15.71:0/3827828067 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.437+0000 7fb0777fe640 1 -- 172.21.15.71:0/3827828067 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c47b0 msgr2=0x7fb0781563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.437+0000 7fb0777fe640 1 --2- 172.21.15.71:0/3827828067 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c47b0 0x7fb0781563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.437+0000 7fb0777fe640 1 -- 172.21.15.71:0/3827828067 >> v1:172.21.15.122:6789/0 conn(0x7fb0781494e0 legacy=0x7fb078153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.437+0000 7fb0777fe640 1 -- 172.21.15.71:0/3827828067 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0780c27c0 con 0x7fb0780c32a0 2024-04-16T15:41:00.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.437+0000 7fb0777fe640 1 --2- 172.21.15.71:0/3827828067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c32a0 0x7fb078155e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0097a0 tx=0x7fb06c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f1cc9b6b9bddb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb076ffd640 1 -- 172.21.15.71:0/3827828067 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0315a0 con 0x7fb0780c32a0 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb076ffd640 1 -- 172.21.15.71:0/3827828067 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06c037070 con 0x7fb0780c32a0 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb076ffd640 1 -- 172.21.15.71:0/3827828067 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c031960 con 0x7fb0780c32a0 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3827828067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c32a0 msgr2=0x7fb078155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3827828067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c32a0 0x7fb078155e60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0097a0 tx=0x7fb06c02b840 comp rx=0 tx=0).stop 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3827828067 shutdown_connections 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3827828067 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c47b0 0x7fb0781563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3827828067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c32a0 0x7fb078155e60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3827828067 >> 172.21.15.71:0/3827828067 conn(0x7fb0780c7940 msgr2=0x7fb0780c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3827828067 shutdown_connections 2024-04-16T15:41:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.438+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3827828067 wait complete. 2024-04-16T15:41:00.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.439+0000 7fb07e2d9640 1 Processor -- start 2024-04-16T15:41:00.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07e2d9640 1 -- start start 2024-04-16T15:41:00.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07e2d9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c32a0 0x7fb0781ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07e2d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c47b0 0x7fb0781eb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07e2d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0781494e0 0x7fb078151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07e2d9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0781544c0 con 0x7fb0781494e0 2024-04-16T15:41:00.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07e2d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0780c2d80 con 0x7fb0780c47b0 2024-04-16T15:41:00.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07e2d9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb0781ef830 con 0x7fb0780c32a0 2024-04-16T15:41:00.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07c876640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0781494e0 0x7fb078151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07c876640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0781494e0 0x7fb078151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54916/0 (socket says 172.21.15.71:54916) 2024-04-16T15:41:00.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.440+0000 7fb07c876640 1 -- 172.21.15.71:0/3175740303 learned_addr learned my addr 172.21.15.71:0/3175740303 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:00.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb077fff640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c32a0 0x7fb0781ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb0777fe640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c47b0 0x7fb0781eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb07c876640 1 -- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c32a0 msgr2=0x7fb0781ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb07c876640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c32a0 0x7fb0781ef2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb07c876640 1 -- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c47b0 msgr2=0x7fb0781eb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb07c876640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c47b0 0x7fb0781eb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb07c876640 1 -- 172.21.15.71:0/3175740303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb078073680 con 0x7fb0781494e0 2024-04-16T15:41:00.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb0777fe640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c47b0 0x7fb0781eb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:00.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb07c876640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0781494e0 0x7fb078151a00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb07000efc0 tx=0x7fb07000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb074ff9640 1 -- 172.21.15.71:0/3175740303 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07000cd20 con 0x7fb0781494e0 2024-04-16T15:41:00.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb074ff9640 1 -- 172.21.15.71:0/3175740303 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb070003990 con 0x7fb0781494e0 2024-04-16T15:41:00.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0781eeaf0 con 0x7fb0781494e0 2024-04-16T15:41:00.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb074ff9640 1 -- 172.21.15.71:0/3175740303 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070003d80 con 0x7fb0781494e0 2024-04-16T15:41:00.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.441+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb078071990 con 0x7fb0781494e0 2024-04-16T15:41:00.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.442+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0781ee8e0 con 0x7fb0781494e0 2024-04-16T15:41:00.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.443+0000 7fb074ff9640 1 -- 172.21.15.71:0/3175740303 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb070012050 con 0x7fb0781494e0 2024-04-16T15:41:00.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.443+0000 7fb074ff9640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.443+0000 7fb074ff9640 1 -- 172.21.15.71:0/3175740303 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb07008c320 con 0x7fb0781494e0 2024-04-16T15:41:00.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.443+0000 7fb077fff640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.443+0000 7fb077fff640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb06406c8f0 0x7fb06406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb0680015a0 tx=0x7fb068009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.447+0000 7fb074ff9640 1 -- 172.21.15.71:0/3175740303 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb070010040 con 0x7fb0781494e0 2024-04-16T15:41:00.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.626+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb078068be0 con 0x7fb0781494e0 2024-04-16T15:41:00.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.626+0000 7fb074ff9640 1 -- 172.21.15.71:0/3175740303 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fb07005a4a0 con 0x7fb0781494e0 2024-04-16T15:41:00.625 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:41:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb06406c8f0 msgr2=0x7fb06406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb06406c8f0 0x7fb06406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb0680015a0 tx=0x7fb068009040 comp rx=0 tx=0).stop 2024-04-16T15:41:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0781494e0 msgr2=0x7fb078151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0781494e0 0x7fb078151a00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb07000efc0 tx=0x7fb07000c490 comp rx=0 tx=0).stop 2024-04-16T15:41:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 shutdown_connections 2024-04-16T15:41:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb06406c8f0 0x7fb06406edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0781494e0 0x7fb078151a00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0780c47b0 0x7fb0781eb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 --2- 172.21.15.71:0/3175740303 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0780c32a0 0x7fb0781ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 >> 172.21.15.71:0/3175740303 conn(0x7fb0780c7940 msgr2=0x7fb07806ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 shutdown_connections 2024-04-16T15:41:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.628+0000 7fb07e2d9640 1 -- 172.21.15.71:0/3175740303 wait complete. 2024-04-16T15:41:00.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:41:00.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.745+0000 7f745a95f640 1 Processor -- start 2024-04-16T15:41:00.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.745+0000 7f745a95f640 1 -- start start 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f745a95f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7454153570 0x7f74540c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f745a95f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f74540c4350 0x7f74540c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f745a95f640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f74540c62e0 0x7f7454159740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f745a95f640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f74540c41a0 con 0x7f74540c62e0 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f745a95f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f74540c82d0 con 0x7f74540c4350 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f745a95f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7454155d70 con 0x7f7454153570 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f7458efc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f74540c62e0 0x7f7454159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f7458efc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f74540c62e0 0x7f7454159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54918/0 (socket says 172.21.15.71:54918) 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f7458efc640 1 -- 172.21.15.71:0/1274083235 learned_addr learned my addr 172.21.15.71:0/1274083235 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f7458efc640 1 -- 172.21.15.71:0/1274083235 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7454153570 msgr2=0x7f74540c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:00.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f7458efc640 1 --2- 172.21.15.71:0/1274083235 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7454153570 0x7f74540c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f7458efc640 1 -- 172.21.15.71:0/1274083235 >> v2:172.21.15.122:3300/0 conn(0x7f74540c4350 msgr2=0x7f74540c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f7458efc640 1 --2- 172.21.15.71:0/1274083235 >> v2:172.21.15.122:3300/0 conn(0x7f74540c4350 0x7f74540c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.746+0000 7f7458efc640 1 -- 172.21.15.71:0/1274083235 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74540c8d50 con 0x7f74540c62e0 2024-04-16T15:41:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.747+0000 7f7458efc640 1 --2- 172.21.15.71:0/1274083235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74540c62e0 0x7f7454159740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f74540c3a00 tx=0x7f744c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20dcf684923d5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.747+0000 7f7452ffd640 1 -- 172.21.15.71:0/1274083235 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c02c950 con 0x7f74540c62e0 2024-04-16T15:41:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.747+0000 7f7452ffd640 1 -- 172.21.15.71:0/1274083235 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f744c02caf0 con 0x7f74540c62e0 2024-04-16T15:41:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.747+0000 7f7452ffd640 1 -- 172.21.15.71:0/1274083235 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c0343f0 con 0x7f74540c62e0 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 -- 172.21.15.71:0/1274083235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74540c62e0 msgr2=0x7f7454159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 --2- 172.21.15.71:0/1274083235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74540c62e0 0x7f7454159740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f74540c3a00 tx=0x7f744c02bdb0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 -- 172.21.15.71:0/1274083235 shutdown_connections 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 --2- 172.21.15.71:0/1274083235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74540c62e0 0x7f7454159740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 --2- 172.21.15.71:0/1274083235 >> v2:172.21.15.122:3300/0 conn(0x7f74540c4350 0x7f74540c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 --2- 172.21.15.71:0/1274083235 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7454153570 0x7f74540c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 -- 172.21.15.71:0/1274083235 >> 172.21.15.71:0/1274083235 conn(0x7f74540c96e0 msgr2=0x7f74540c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 -- 172.21.15.71:0/1274083235 shutdown_connections 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.748+0000 7f745a95f640 1 -- 172.21.15.71:0/1274083235 wait complete. 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.749+0000 7f745a95f640 1 Processor -- start 2024-04-16T15:41:00.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.749+0000 7f745a95f640 1 -- start start 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f745a95f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74540c4350 0x7f74540abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f745a95f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f74540c62e0 0x7f745414e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f745a95f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454153570 0x7f74541ec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f745a95f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7454159c80 con 0x7f7454153570 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f745a95f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74540c8ff0 con 0x7f74540c4350 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f745a95f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7454159e00 con 0x7f74540c62e0 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454153570 0x7f74541ec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454153570 0x7f74541ec3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54932/0 (socket says 172.21.15.71:54932) 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 -- 172.21.15.71:0/993417459 learned_addr learned my addr 172.21.15.71:0/993417459 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7453fff640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74540c4350 0x7f74540abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f74537fe640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f74540c62e0 0x7f745414e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 -- 172.21.15.71:0/993417459 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f74540c62e0 msgr2=0x7f745414e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f74540c62e0 0x7f745414e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 -- 172.21.15.71:0/993417459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74540c4350 msgr2=0x7f74540abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74540c4350 0x7f74540abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 -- 172.21.15.71:0/993417459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745414ee20 con 0x7f7454153570 2024-04-16T15:41:00.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7458efc640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454153570 0x7f74541ec3e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f744c009f40 tx=0x7f744c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.750+0000 7f7450ff9640 1 -- 172.21.15.71:0/993417459 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c043020 con 0x7f7454153570 2024-04-16T15:41:00.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.751+0000 7f7450ff9640 1 -- 172.21.15.71:0/993417459 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f744c03e450 con 0x7f7454153570 2024-04-16T15:41:00.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.751+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7454150d70 con 0x7f7454153570 2024-04-16T15:41:00.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.751+0000 7f7450ff9640 1 -- 172.21.15.71:0/993417459 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c03e820 con 0x7f7454153570 2024-04-16T15:41:00.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.751+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7454151330 con 0x7f7454153570 2024-04-16T15:41:00.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.751+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7454151530 con 0x7f7454153570 2024-04-16T15:41:00.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.752+0000 7f7450ff9640 1 -- 172.21.15.71:0/993417459 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f744c046020 con 0x7f7454153570 2024-04-16T15:41:00.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.752+0000 7f7450ff9640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f744006c920 0x7f744006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:00.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.752+0000 7f7450ff9640 1 -- 172.21.15.71:0/993417459 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f744c0ae360 con 0x7f7454153570 2024-04-16T15:41:00.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.753+0000 7f7453fff640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f744006c920 0x7f744006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:00.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.753+0000 7f7453fff640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f744006c920 0x7f744006ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74540c6050 tx=0x7f7444009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:00.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.757+0000 7f7450ff9640 1 -- 172.21.15.71:0/993417459 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f744c04d070 con 0x7f7454153570 2024-04-16T15:41:00.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.934+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7454068be0 con 0x7f7454153570 2024-04-16T15:41:00.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.935+0000 7f7450ff9640 1 -- 172.21.15.71:0/993417459 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f745414ee20 con 0x7f7454153570 2024-04-16T15:41:00.934 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f744006c920 msgr2=0x7f744006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f744006c920 0x7f744006ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74540c6050 tx=0x7f7444009060 comp rx=0 tx=0).stop 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454153570 msgr2=0x7f74541ec3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454153570 0x7f74541ec3e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f744c009f40 tx=0x7f744c004a40 comp rx=0 tx=0).stop 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 shutdown_connections 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f744006c920 0x7f744006ede0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454153570 0x7f74541ec3e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f74540c62e0 0x7f745414e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 --2- 172.21.15.71:0/993417459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74540c4350 0x7f74540abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 >> 172.21.15.71:0/993417459 conn(0x7f74540c96e0 msgr2=0x7f74540c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 shutdown_connections 2024-04-16T15:41:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:00.937+0000 7f745a95f640 1 -- 172.21.15.71:0/993417459 wait complete. 2024-04-16T15:41:00.971 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) That unsetting mds_join_fs will not cause failover if no high-affinity standby exists. 2024-04-16T15:41:00.971 INFO:tasks.cephfs_test_runner:test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) 2024-04-16T15:41:00.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:01.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.077+0000 7f713bb52640 1 Processor -- start 2024-04-16T15:41:01.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.077+0000 7f713bb52640 1 -- start start 2024-04-16T15:41:01.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f713bb52640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f71341531a0 0x7f7134159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f713bb52640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c26c0 0x7f71340c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f713bb52640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71340c4680 con 0x7f71340c1c90 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f713bb52640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f71340c38f0 con 0x7f71341531a0 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f713bb52640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7134071110 con 0x7f71340c26c0 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71390ed640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c26c0 0x7f71340c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71398ee640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f71341531a0 0x7f7134159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71390ed640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c26c0 0x7f71340c1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:55332/0 (socket says 172.21.15.71:55332) 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f713a0ef640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f71340c1c90 0x7f713406f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:54222/0 (socket says 172.21.15.71:54222) 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71390ed640 1 -- 172.21.15.71:0/2748406259 learned_addr learned my addr 172.21.15.71:0/2748406259 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71388ec640 1 -- 172.21.15.71:0/2748406259 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022551996 0 0) 0x7f71340c4680 con 0x7f71340c1c90 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71388ec640 1 -- 172.21.15.71:0/2748406259 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7120003520 con 0x7f71340c1c90 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71390ed640 1 -- 172.21.15.71:0/2748406259 >> v2:172.21.15.122:3300/0 conn(0x7f71341531a0 msgr2=0x7f7134159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71390ed640 1 --2- 172.21.15.71:0/2748406259 >> v2:172.21.15.122:3300/0 conn(0x7f71341531a0 0x7f7134159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71390ed640 1 -- 172.21.15.71:0/2748406259 >> v1:172.21.15.122:6789/0 conn(0x7f71340c1c90 legacy=0x7f713406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71390ed640 1 -- 172.21.15.71:0/2748406259 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71340c4960 con 0x7f71340c26c0 2024-04-16T15:41:01.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.078+0000 7f71398ee640 1 --2- 172.21.15.71:0/2748406259 >> v2:172.21.15.122:3300/0 conn(0x7f71341531a0 0x7f7134159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.079+0000 7f71390ed640 1 --2- 172.21.15.71:0/2748406259 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c26c0 0x7f71340c1750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7124009870 tx=0x7f712402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7171deae0fdf24e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.079+0000 7f71388ec640 1 -- 172.21.15.71:0/2748406259 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124034070 con 0x7f71340c26c0 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.079+0000 7f71388ec640 1 -- 172.21.15.71:0/2748406259 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124030840 con 0x7f71340c26c0 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.079+0000 7f71388ec640 1 -- 172.21.15.71:0/2748406259 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124030bf0 con 0x7f71340c26c0 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.080+0000 7f713bb52640 1 -- 172.21.15.71:0/2748406259 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c26c0 msgr2=0x7f71340c1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.080+0000 7f713bb52640 1 --2- 172.21.15.71:0/2748406259 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c26c0 0x7f71340c1750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7124009870 tx=0x7f712402b980 comp rx=0 tx=0).stop 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.080+0000 7f713bb52640 1 -- 172.21.15.71:0/2748406259 shutdown_connections 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.080+0000 7f713bb52640 1 --2- 172.21.15.71:0/2748406259 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c26c0 0x7f71340c1750 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.080+0000 7f713bb52640 1 --2- 172.21.15.71:0/2748406259 >> v2:172.21.15.122:3300/0 conn(0x7f71341531a0 0x7f7134159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.080+0000 7f713bb52640 1 -- 172.21.15.71:0/2748406259 >> 172.21.15.71:0/2748406259 conn(0x7f71340c4e20 msgr2=0x7f71340c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:01.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.080+0000 7f713bb52640 1 -- 172.21.15.71:0/2748406259 shutdown_connections 2024-04-16T15:41:01.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.080+0000 7f713bb52640 1 -- 172.21.15.71:0/2748406259 wait complete. 2024-04-16T15:41:01.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.081+0000 7f713bb52640 1 Processor -- start 2024-04-16T15:41:01.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.081+0000 7f713bb52640 1 -- start start 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f713bb52640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c1c90 0x7f71341e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f713bb52640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71340c26c0 0x7f71340c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f713bb52640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71341531a0 0x7f713414a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f713bb52640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f713406ff70 con 0x7f71340c26c0 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f713bb52640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7134153580 con 0x7f71341531a0 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f713bb52640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f71340700f0 con 0x7f71340c1c90 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71390ed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71340c26c0 0x7f71340c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71390ed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71340c26c0 0x7f71340c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54938/0 (socket says 172.21.15.71:54938) 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71390ed640 1 -- 172.21.15.71:0/1213980429 learned_addr learned my addr 172.21.15.71:0/1213980429 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71398ee640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c1c90 0x7f71341e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f713a0ef640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71341531a0 0x7f713414a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71390ed640 1 -- 172.21.15.71:0/1213980429 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c1c90 msgr2=0x7f71341e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71390ed640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c1c90 0x7f71341e88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71390ed640 1 -- 172.21.15.71:0/1213980429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71341531a0 msgr2=0x7f713414a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71390ed640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71341531a0 0x7f713414a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.082+0000 7f71390ed640 1 -- 172.21.15.71:0/1213980429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71341ef2d0 con 0x7f71340c26c0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.083+0000 7f71390ed640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71340c26c0 0x7f71340c71d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7124000c00 tx=0x7f7124004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.083+0000 7f712a7fc640 1 -- 172.21.15.71:0/1213980429 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124048070 con 0x7f71340c26c0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.083+0000 7f712a7fc640 1 -- 172.21.15.71:0/1213980429 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124039a90 con 0x7f71340c26c0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.083+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71341ef5b0 con 0x7f71340c26c0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.083+0000 7f712a7fc640 1 -- 172.21.15.71:0/1213980429 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124034040 con 0x7f71340c26c0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.083+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71340ad7e0 con 0x7f71340c26c0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.084+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71341ef780 con 0x7f71340c26c0 2024-04-16T15:41:01.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.084+0000 7f712a7fc640 1 -- 172.21.15.71:0/1213980429 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7124037070 con 0x7f71340c26c0 2024-04-16T15:41:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.085+0000 7f712a7fc640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f711806c920 0x7f711806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.085+0000 7f71398ee640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f711806c920 0x7f711806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.085+0000 7f712a7fc640 1 -- 172.21.15.71:0/1213980429 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f71240ad8d0 con 0x7f71340c26c0 2024-04-16T15:41:01.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.085+0000 7f71398ee640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f711806c920 0x7f711806ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7130009560 tx=0x7f7130009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.089+0000 7f712a7fc640 1 -- 172.21.15.71:0/1213980429 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f712407b380 con 0x7f71340c26c0 2024-04-16T15:41:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.264+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7134068cf0 con 0x7f71340c26c0 2024-04-16T15:41:01.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.266+0000 7f712a7fc640 1 -- 172.21.15.71:0/1213980429 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18502 (secure 0 0 0) 0x7f712407f680 con 0x7f71340c26c0 2024-04-16T15:41:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:01.265 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:40:28.915391+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f711806c920 msgr2=0x7f711806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f711806c920 0x7f711806ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7130009560 tx=0x7f7130009190 comp rx=0 tx=0).stop 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71340c26c0 msgr2=0x7f71340c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71340c26c0 0x7f71340c71d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7124000c00 tx=0x7f7124004730 comp rx=0 tx=0).stop 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 shutdown_connections 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f711806c920 0x7f711806ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71341531a0 0x7f713414a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71340c26c0 0x7f71340c71d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 --2- 172.21.15.71:0/1213980429 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71340c1c90 0x7f71341e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 >> 172.21.15.71:0/1213980429 conn(0x7f71340c4e20 msgr2=0x7f713406cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 shutdown_connections 2024-04-16T15:41:01.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.268+0000 7f713bb52640 1 -- 172.21.15.71:0/1213980429 wait complete. 2024-04-16T15:41:01.279 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:41:01.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:41:01.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.387+0000 7f4d54f50640 1 Processor -- start 2024-04-16T15:41:01.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.387+0000 7f4d54f50640 1 -- start start 2024-04-16T15:41:01.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.388+0000 7f4d54f50640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4d5014da40 0x7f4d500ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.388+0000 7f4d54f50640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f4d500aca10 0x7f4d500c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.388+0000 7f4d54f50640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5014de20 con 0x7f4d500c3fb0 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.388+0000 7f4d54f50640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f4d500c9120 con 0x7f4d500aca10 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.388+0000 7f4d54f50640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4d501548b0 con 0x7f4d5014da40 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.388+0000 7f4d4dd9b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f4d500aca10 0x7f4d500c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.388+0000 7f4d4dd9b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f4d500aca10 0x7f4d500c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54952/0 (socket says 172.21.15.71:54952) 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.388+0000 7f4d4dd9b640 1 -- 172.21.15.71:0/3369364208 learned_addr learned my addr 172.21.15.71:0/3369364208 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.389+0000 7f4d4dd9b640 1 -- 172.21.15.71:0/3369364208 >> v2:172.21.15.122:3300/0 conn(0x7f4d5014da40 msgr2=0x7f4d500ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.389+0000 7f4d4e59c640 1 --2- 172.21.15.71:0/3369364208 >> v2:172.21.15.122:3300/0 conn(0x7f4d5014da40 0x7f4d500ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.389+0000 7f4d4dd9b640 1 --2- 172.21.15.71:0/3369364208 >> v2:172.21.15.122:3300/0 conn(0x7f4d5014da40 0x7f4d500ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.389+0000 7f4d4dd9b640 1 -- 172.21.15.71:0/3369364208 >> v1:172.21.15.122:6789/0 conn(0x7f4d500c3fb0 legacy=0x7f4d50156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.389+0000 7f4d4dd9b640 1 -- 172.21.15.71:0/3369364208 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d500ae840 con 0x7f4d500aca10 2024-04-16T15:41:01.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.389+0000 7f4d4e59c640 1 --2- 172.21.15.71:0/3369364208 >> v2:172.21.15.122:3300/0 conn(0x7f4d5014da40 0x7f4d500ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d4dd9b640 1 --2- 172.21.15.71:0/3369364208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d500c78e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009870 tx=0x7f4d3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5d4919435ff124 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d4d59a640 1 -- 172.21.15.71:0/3369364208 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38035020 con 0x7f4d500aca10 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d4d59a640 1 -- 172.21.15.71:0/3369364208 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d38031870 con 0x7f4d500aca10 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d4d59a640 1 -- 172.21.15.71:0/3369364208 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38031c40 con 0x7f4d500aca10 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d54f50640 1 -- 172.21.15.71:0/3369364208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 msgr2=0x7f4d500c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d54f50640 1 --2- 172.21.15.71:0/3369364208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d500c78e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009870 tx=0x7f4d3802b910 comp rx=0 tx=0).stop 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d54f50640 1 -- 172.21.15.71:0/3369364208 shutdown_connections 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d54f50640 1 --2- 172.21.15.71:0/3369364208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d500c78e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d54f50640 1 --2- 172.21.15.71:0/3369364208 >> v2:172.21.15.122:3300/0 conn(0x7f4d5014da40 0x7f4d500ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d54f50640 1 -- 172.21.15.71:0/3369364208 >> 172.21.15.71:0/3369364208 conn(0x7f4d500c9420 msgr2=0x7f4d500c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d54f50640 1 -- 172.21.15.71:0/3369364208 shutdown_connections 2024-04-16T15:41:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.390+0000 7f4d54f50640 1 -- 172.21.15.71:0/3369364208 wait complete. 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d54f50640 1 Processor -- start 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d54f50640 1 -- start start 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d54f50640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d501eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d54f50640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4d500c3fb0 0x7f4d50149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d54f50640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d5014da40 0x7f4d5014a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d54f50640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4d500c4a10 con 0x7f4d500aca10 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d54f50640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d500c73c0 con 0x7f4d5014da40 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d54f50640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4d500c39f0 con 0x7f4d500c3fb0 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d4e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d501eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d4e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d501eccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54962/0 (socket says 172.21.15.71:54962) 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.392+0000 7f4d4e59c640 1 -- 172.21.15.71:0/1479649949 learned_addr learned my addr 172.21.15.71:0/1479649949 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:01.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d4e59c640 1 -- 172.21.15.71:0/1479649949 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4d500c3fb0 msgr2=0x7f4d50149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d4ed9d640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d5014da40 0x7f4d5014a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d4e59c640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4d500c3fb0 0x7f4d50149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d4e59c640 1 -- 172.21.15.71:0/1479649949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d5014da40 msgr2=0x7f4d5014a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d4e59c640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d5014da40 0x7f4d5014a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d4e59c640 1 -- 172.21.15.71:0/1479649949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d501ed9a0 con 0x7f4d500aca10 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d4ed9d640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d5014da40 0x7f4d5014a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d4e59c640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d501eccd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000cf90 tx=0x7f4d4000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d46ffd640 1 -- 172.21.15.71:0/1479649949 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4001a070 con 0x7f4d500aca10 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d46ffd640 1 -- 172.21.15.71:0/1479649949 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4000d840 con 0x7f4d500aca10 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d501467d0 con 0x7f4d500aca10 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d46ffd640 1 -- 172.21.15.71:0/1479649949 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000dc10 con 0x7f4d500aca10 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.393+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d50150c70 con 0x7f4d500aca10 2024-04-16T15:41:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.394+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d500af7c0 con 0x7f4d500aca10 2024-04-16T15:41:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.395+0000 7f4d46ffd640 1 -- 172.21.15.71:0/1479649949 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d4001d070 con 0x7f4d500aca10 2024-04-16T15:41:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.395+0000 7f4d46ffd640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4d3006c8f0 0x7f4d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.395+0000 7f4d46ffd640 1 -- 172.21.15.71:0/1479649949 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4d4008c5f0 con 0x7f4d500aca10 2024-04-16T15:41:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.395+0000 7f4d4dd9b640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4d3006c8f0 0x7f4d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.396+0000 7f4d4dd9b640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4d3006c8f0 0x7f4d3006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d38036000 tx=0x7f4d38030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.399+0000 7f4d46ffd640 1 -- 172.21.15.71:0/1479649949 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d500af7c0 con 0x7f4d500aca10 2024-04-16T15:41:01.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.574+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d50068be0 con 0x7f4d500aca10 2024-04-16T15:41:01.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.575+0000 7f4d46ffd640 1 -- 172.21.15.71:0/1479649949 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f4d4005a0a0 con 0x7f4d500aca10 2024-04-16T15:41:01.574 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4d3006c8f0 msgr2=0x7f4d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4d3006c8f0 0x7f4d3006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d38036000 tx=0x7f4d38030000 comp rx=0 tx=0).stop 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 msgr2=0x7f4d501eccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d501eccd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000cf90 tx=0x7f4d4000a7d0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 shutdown_connections 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f4d3006c8f0 0x7f4d3006edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d5014da40 0x7f4d5014a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4d500c3fb0 0x7f4d50149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 --2- 172.21.15.71:0/1479649949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d500aca10 0x7f4d501eccd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 >> 172.21.15.71:0/1479649949 conn(0x7f4d500c9420 msgr2=0x7f4d500ac110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:01.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 shutdown_connections 2024-04-16T15:41:01.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.577+0000 7f4d54f50640 1 -- 172.21.15.71:0/1479649949 wait complete. 2024-04-16T15:41:01.587 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:41:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.694+0000 7f0dcf73d640 1 Processor -- start 2024-04-16T15:41:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.694+0000 7f0dcf73d640 1 -- start start 2024-04-16T15:41:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcf73d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0dc8155dc0 0x7f0dc80c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcf73d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc80c27c0 con 0x7f0dc80ab290 2024-04-16T15:41:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcf73d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc80c2aa0 con 0x7f0dc806fac0 2024-04-16T15:41:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcf73d640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc8073f20 con 0x7f0dc8155dc0 2024-04-16T15:41:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcd4d9640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0dc8155dc0 0x7f0dc80c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcdcda640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f0dc80ab290 0x7f0dc815d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33556/0 (socket says 172.21.15.71:33556) 2024-04-16T15:41:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcdcda640 1 -- 172.21.15.71:0/1306704179 learned_addr learned my addr 172.21.15.71:0/1306704179 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcd4d9640 1 -- 172.21.15.71:0/1306704179 >> v1:172.21.15.122:6789/0 conn(0x7f0dc806fac0 legacy=0x7f0dc81561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcd4d9640 1 -- 172.21.15.71:0/1306704179 >> v1:172.21.15.71:6789/0 conn(0x7f0dc80ab290 legacy=0x7f0dc815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dcd4d9640 1 -- 172.21.15.71:0/1306704179 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc80c2d80 con 0x7f0dc8155dc0 2024-04-16T15:41:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.695+0000 7f0dbffff640 1 -- 172.21.15.71:0/1306704179 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2650011488 0 0) 0x7f0dc80c27c0 con 0x7f0dc80ab290 2024-04-16T15:41:01.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.696+0000 7f0dcd4d9640 1 --2- 172.21.15.71:0/1306704179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc8155dc0 0x7f0dc80c32a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40097a0 tx=0x7f0dc402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07c211208d0876c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.696+0000 7f0dbffff640 1 -- 172.21.15.71:0/1306704179 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4034070 con 0x7f0dc8155dc0 2024-04-16T15:41:01.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.696+0000 7f0dbffff640 1 -- 172.21.15.71:0/1306704179 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc4030610 con 0x7f0dc8155dc0 2024-04-16T15:41:01.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.696+0000 7f0dbffff640 1 -- 172.21.15.71:0/1306704179 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40309e0 con 0x7f0dc8155dc0 2024-04-16T15:41:01.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.696+0000 7f0dcf73d640 1 -- 172.21.15.71:0/1306704179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc8155dc0 msgr2=0x7f0dc80c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.696+0000 7f0dcf73d640 1 --2- 172.21.15.71:0/1306704179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc8155dc0 0x7f0dc80c32a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40097a0 tx=0x7f0dc402b960 comp rx=0 tx=0).stop 2024-04-16T15:41:01.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.697+0000 7f0dcf73d640 1 -- 172.21.15.71:0/1306704179 shutdown_connections 2024-04-16T15:41:01.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.697+0000 7f0dcf73d640 1 --2- 172.21.15.71:0/1306704179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc8155dc0 0x7f0dc80c32a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.697+0000 7f0dcf73d640 1 -- 172.21.15.71:0/1306704179 >> 172.21.15.71:0/1306704179 conn(0x7f0dc80c3e10 msgr2=0x7f0dc80c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:01.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.697+0000 7f0dcf73d640 1 -- 172.21.15.71:0/1306704179 shutdown_connections 2024-04-16T15:41:01.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.697+0000 7f0dcf73d640 1 -- 172.21.15.71:0/1306704179 wait complete. 2024-04-16T15:41:01.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcf73d640 1 Processor -- start 2024-04-16T15:41:01.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcf73d640 1 -- start start 2024-04-16T15:41:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcf73d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc80ab290 0x7f0dc8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcf73d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8155dc0 0x7f0dc806c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcf73d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc81dd490 0x7f0dc81eb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcf73d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc806bd70 con 0x7f0dc80ab290 2024-04-16T15:41:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcf73d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8156ab0 con 0x7f0dc81dd490 2024-04-16T15:41:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcf73d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc806bef0 con 0x7f0dc8155dc0 2024-04-16T15:41:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcd4d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc80ab290 0x7f0dc8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcd4d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc80ab290 0x7f0dc8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54972/0 (socket says 172.21.15.71:54972) 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.698+0000 7f0dcd4d9640 1 -- 172.21.15.71:0/3598496969 learned_addr learned my addr 172.21.15.71:0/3598496969 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcdcda640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc81dd490 0x7f0dc81eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcd4d9640 1 -- 172.21.15.71:0/3598496969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8155dc0 msgr2=0x7f0dc806c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcd4d9640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8155dc0 0x7f0dc806c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcd4d9640 1 -- 172.21.15.71:0/3598496969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc81dd490 msgr2=0x7f0dc81eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcd4d9640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc81dd490 0x7f0dc81eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcd4d9640 1 -- 172.21.15.71:0/3598496969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc81dc360 con 0x7f0dc80ab290 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcd4d9640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc80ab290 0x7f0dc8073090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40098c0 tx=0x7f0dc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dbdffb640 1 -- 172.21.15.71:0/3598496969 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4034040 con 0x7f0dc80ab290 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dbdffb640 1 -- 172.21.15.71:0/3598496969 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc403acc0 con 0x7f0dc80ab290 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc81dc640 con 0x7f0dc80ab290 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dbdffb640 1 -- 172.21.15.71:0/3598496969 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4044630 con 0x7f0dc80ab290 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.699+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc806c970 con 0x7f0dc80ab290 2024-04-16T15:41:01.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.700+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc81dc860 con 0x7f0dc80ab290 2024-04-16T15:41:01.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.701+0000 7f0dbdffb640 1 -- 172.21.15.71:0/3598496969 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0dc404b060 con 0x7f0dc80ab290 2024-04-16T15:41:01.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.701+0000 7f0dbdffb640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0dac06c8f0 0x7f0dac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:01.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.701+0000 7f0dbdffb640 1 -- 172.21.15.71:0/3598496969 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0dc40adde0 con 0x7f0dc80ab290 2024-04-16T15:41:01.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.701+0000 7f0dcccd8640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0dac06c8f0 0x7f0dac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:01.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.702+0000 7f0dcccd8640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0dac06c8f0 0x7f0dac06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0db8000f60 tx=0x7f0db80079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:01.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.706+0000 7f0dbdffb640 1 -- 172.21.15.71:0/3598496969 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0dc403f070 con 0x7f0dc80ab290 2024-04-16T15:41:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.882+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0dc806bb50 con 0x7f0dc80ab290 2024-04-16T15:41:01.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.882+0000 7f0dbdffb640 1 -- 172.21.15.71:0/3598496969 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f0dc404b350 con 0x7f0dc80ab290 2024-04-16T15:41:01.881 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0dac06c8f0 msgr2=0x7f0dac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0dac06c8f0 0x7f0dac06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0db8000f60 tx=0x7f0db80079b0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc80ab290 msgr2=0x7f0dc8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc80ab290 0x7f0dc8073090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40098c0 tx=0x7f0dc40046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 shutdown_connections 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0dac06c8f0 0x7f0dac06edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc81dd490 0x7f0dc81eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8155dc0 0x7f0dc806c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 --2- 172.21.15.71:0/3598496969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0dc80ab290 0x7f0dc8073090 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.884+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 >> 172.21.15.71:0/3598496969 conn(0x7f0dc80c3e10 msgr2=0x7f0dc81de730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.885+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 shutdown_connections 2024-04-16T15:41:01.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:01.885+0000 7f0dcf73d640 1 -- 172.21.15.71:0/3598496969 wait complete. 2024-04-16T15:41:01.894 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:41:02.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.001+0000 7f1b24247640 1 Processor -- start 2024-04-16T15:41:02.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.001+0000 7f1b24247640 1 -- start start 2024-04-16T15:41:02.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.002+0000 7f1b24247640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1b1c14ffe0 0x7f1b1c155e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.002+0000 7f1b24247640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c06cc60 con 0x7f1b1c154450 2024-04-16T15:41:02.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.002+0000 7f1b24247640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c1569f0 con 0x7f1b1c06cec0 2024-04-16T15:41:02.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.002+0000 7f1b24247640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1c0c2720 con 0x7f1b1c14ffe0 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.002+0000 7f1b227e4640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f1b1c154450 0x7f1b1c15d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33558/0 (socket says 172.21.15.71:33558) 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.002+0000 7f1b227e4640 1 -- 172.21.15.71:0/3488899134 learned_addr learned my addr 172.21.15.71:0/3488899134 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 100576209 0 0) 0x7f1b1c06cc60 con 0x7f1b1c154450 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b08003520 con 0x7f1b1c154450 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/3488899134 >> v2:172.21.15.122:3300/0 conn(0x7f1b1c14ffe0 0x7f1b1c155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2010056954 0 0) 0x7f1b08003520 con 0x7f1b1c154450 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 >> v2:172.21.15.122:3300/0 conn(0x7f1b1c14ffe0 msgr2=0x7f1b1c155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 --2- 172.21.15.71:0/3488899134 >> v2:172.21.15.122:3300/0 conn(0x7f1b1c14ffe0 0x7f1b1c155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 >> v1:172.21.15.122:6789/0 conn(0x7f1b1c06cec0 legacy=0x7f1b1c153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c0c2970 con 0x7f1b1c154450 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/3488899134 >> v2:172.21.15.122:3300/0 conn(0x7f1b1c14ffe0 0x7f1b1c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f1b14001670 con 0x7f1b1c154450 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 >> v1:172.21.15.71:6789/0 conn(0x7f1b1c154450 legacy=0x7f1b1c15d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b080039b0 0x7f1b08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b08004800 0x7f1b08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b080052e0 0x7f1b0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c0c2720 con 0x7f1b080039b0 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0800c1b0 con 0x7f1b08004800 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0800c3b0 con 0x7f1b080052e0 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b080039b0 0x7f1b08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b217e2640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b08004800 0x7f1b08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b227e4640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b080052e0 0x7f1b0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b21fe3640 1 -- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b080052e0 msgr2=0x7f1b0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b080052e0 0x7f1b0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b21fe3640 1 -- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b08004800 msgr2=0x7f1b08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b08004800 0x7f1b08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b21fe3640 1 -- 172.21.15.71:0/3488899134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b08003520 con 0x7f1b080039b0 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b227e4640 1 -- 172.21.15.71:0/3488899134 reap_dead start 2024-04-16T15:41:02.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.003+0000 7f1b217e2640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b08004800 0x7f1b08004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.004+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b080039b0 0x7f1b08003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800bda0 tx=0x7f1b08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bb61d6ad3e63bd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.004+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18034070 con 0x7f1b080039b0 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.004+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b18032070 con 0x7f1b080039b0 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.004+0000 7f1b20fe1640 1 -- 172.21.15.71:0/3488899134 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800b9a0 con 0x7f1b080039b0 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.005+0000 7f1b24247640 1 -- 172.21.15.71:0/3488899134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b080039b0 msgr2=0x7f1b08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.005+0000 7f1b24247640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b080039b0 0x7f1b08003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800bda0 tx=0x7f1b08005830 comp rx=0 tx=0).stop 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.005+0000 7f1b24247640 1 -- 172.21.15.71:0/3488899134 shutdown_connections 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.005+0000 7f1b24247640 1 --2- 172.21.15.71:0/3488899134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b080039b0 0x7f1b08003dd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.005+0000 7f1b24247640 1 -- 172.21.15.71:0/3488899134 >> 172.21.15.71:0/3488899134 conn(0x7f1b1c0c7940 msgr2=0x7f1b1c0c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.005+0000 7f1b24247640 1 -- 172.21.15.71:0/3488899134 shutdown_connections 2024-04-16T15:41:02.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.005+0000 7f1b24247640 1 -- 172.21.15.71:0/3488899134 wait complete. 2024-04-16T15:41:02.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.006+0000 7f1b24247640 1 Processor -- start 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.006+0000 7f1b24247640 1 -- start start 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b24247640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b080039b0 0x7f1b1c0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b24247640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1c0c6110 0x7f1b1c1dde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b24247640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b1c0c0740 0x7f1b1c0c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b24247640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c0c4740 con 0x7f1b1c0c6110 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b24247640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c0c2fc0 con 0x7f1b080039b0 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b24247640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1c0c3450 con 0x7f1b1c0c0740 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1c0c6110 0x7f1b1c1dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1c0c6110 0x7f1b1c1dde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54992/0 (socket says 172.21.15.71:54992) 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 -- 172.21.15.71:0/2019458975 learned_addr learned my addr 172.21.15.71:0/2019458975 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b227e4640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b1c0c0740 0x7f1b1c0c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 -- 172.21.15.71:0/2019458975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b1c0c0740 msgr2=0x7f1b1c0c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b080039b0 0x7f1b1c0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b1c0c0740 0x7f1b1c0c0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 -- 172.21.15.71:0/2019458975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b080039b0 msgr2=0x7f1b1c0c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b080039b0 0x7f1b1c0c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 -- 172.21.15.71:0/2019458975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c1ec500 con 0x7f1b1c0c6110 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b217e2640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1c0c6110 0x7f1b1c1dde80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e4b0 tx=0x7f1b0c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b127fc640 1 -- 172.21.15.71:0/2019458975 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0052e0 con 0x7f1b1c0c6110 2024-04-16T15:41:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b127fc640 1 -- 172.21.15.71:0/2019458975 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0c018070 con 0x7f1b1c0c6110 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c0c54e0 con 0x7f1b1c0c6110 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b127fc640 1 -- 172.21.15.71:0/2019458975 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c003830 con 0x7f1b1c0c6110 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.007+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1eba20 con 0x7f1b1c0c6110 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.008+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c068be0 con 0x7f1b1c0c6110 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.009+0000 7f1b127fc640 1 -- 172.21.15.71:0/2019458975 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b0c0039d0 con 0x7f1b1c0c6110 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.009+0000 7f1b127fc640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.009+0000 7f1b127fc640 1 -- 172.21.15.71:0/2019458975 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1b0c08ba10 con 0x7f1b1c0c6110 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.009+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.010+0000 7f1b21fe3640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1b0006c8f0 0x7f1b0006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1b08004780 tx=0x7f1b1800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.014+0000 7f1b127fc640 1 -- 172.21.15.71:0/2019458975 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b0c059b90 con 0x7f1b1c0c6110 2024-04-16T15:41:02.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.188+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1b1c14fa00 con 0x7f1b1c0c6110 2024-04-16T15:41:02.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.188+0000 7f1b127fc640 1 -- 172.21.15.71:0/2019458975 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f1b0c059490 con 0x7f1b1c0c6110 2024-04-16T15:41:02.187 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:41:02.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1b0006c8f0 msgr2=0x7f1b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1b0006c8f0 0x7f1b0006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1b08004780 tx=0x7f1b1800b040 comp rx=0 tx=0).stop 2024-04-16T15:41:02.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1c0c6110 msgr2=0x7f1b1c1dde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1c0c6110 0x7f1b1c1dde80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e4b0 tx=0x7f1b0c00e970 comp rx=0 tx=0).stop 2024-04-16T15:41:02.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 shutdown_connections 2024-04-16T15:41:02.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1b0006c8f0 0x7f1b0006edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1b1c0c0740 0x7f1b1c0c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1c0c6110 0x7f1b1c1dde80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 --2- 172.21.15.71:0/2019458975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b080039b0 0x7f1b1c0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 >> 172.21.15.71:0/2019458975 conn(0x7f1b1c0c7940 msgr2=0x7f1b1c14e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:02.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 shutdown_connections 2024-04-16T15:41:02.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.190+0000 7f1b24247640 1 -- 172.21.15.71:0/2019458975 wait complete. 2024-04-16T15:41:02.200 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:41:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.307+0000 7fba9bb1c640 1 Processor -- start 2024-04-16T15:41:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.307+0000 7fba9bb1c640 1 -- start start 2024-04-16T15:41:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba9bb1c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fba940b0f40 0x7fba940c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba9bb1c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940c9ad0 0x7fba9406cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba9bb1c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fba94156820 0x7fba9406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba9bb1c640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fba94156cc0 con 0x7fba940b0f40 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba9bb1c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fba940c8170 con 0x7fba94156820 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba9bb1c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fba940c8450 con 0x7fba940c9ad0 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba998b8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fba940b0f40 0x7fba940c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba998b8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fba940b0f40 0x7fba940c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54996/0 (socket says 172.21.15.71:54996) 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.308+0000 7fba998b8640 1 -- 172.21.15.71:0/2470761177 learned_addr learned my addr 172.21.15.71:0/2470761177 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.309+0000 7fba998b8640 1 -- 172.21.15.71:0/2470761177 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940c9ad0 msgr2=0x7fba9406cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.309+0000 7fba990b7640 1 --2- 172.21.15.71:0/2470761177 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940c9ad0 0x7fba9406cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.309+0000 7fba9a0b9640 1 --2- 172.21.15.71:0/2470761177 >> v2:172.21.15.122:3300/0 conn(0x7fba94156820 0x7fba9406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.309+0000 7fba998b8640 1 --2- 172.21.15.71:0/2470761177 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940c9ad0 0x7fba9406cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.309+0000 7fba998b8640 1 -- 172.21.15.71:0/2470761177 >> v2:172.21.15.122:3300/0 conn(0x7fba94156820 msgr2=0x7fba9406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.309+0000 7fba998b8640 1 --2- 172.21.15.71:0/2470761177 >> v2:172.21.15.122:3300/0 conn(0x7fba94156820 0x7fba9406d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.309+0000 7fba998b8640 1 -- 172.21.15.71:0/2470761177 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba940c8730 con 0x7fba940b0f40 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.309+0000 7fba998b8640 1 --2- 172.21.15.71:0/2470761177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba940b0f40 0x7fba940c6040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fba9408d2f0 tx=0x7fba9002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84154db294ed2020 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba988b6640 1 -- 172.21.15.71:0/2470761177 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9002c9e0 con 0x7fba940b0f40 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba988b6640 1 -- 172.21.15.71:0/2470761177 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9002cb80 con 0x7fba940b0f40 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba988b6640 1 -- 172.21.15.71:0/2470761177 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba900343f0 con 0x7fba940b0f40 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 -- 172.21.15.71:0/2470761177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba940b0f40 msgr2=0x7fba940c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/2470761177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba940b0f40 0x7fba940c6040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fba9408d2f0 tx=0x7fba9002be40 comp rx=0 tx=0).stop 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 -- 172.21.15.71:0/2470761177 shutdown_connections 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/2470761177 >> v2:172.21.15.122:3300/0 conn(0x7fba94156820 0x7fba9406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/2470761177 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940c9ad0 0x7fba9406cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/2470761177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba940b0f40 0x7fba940c6040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 -- 172.21.15.71:0/2470761177 >> 172.21.15.71:0/2470761177 conn(0x7fba940ad3f0 msgr2=0x7fba940ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:02.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 -- 172.21.15.71:0/2470761177 shutdown_connections 2024-04-16T15:41:02.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.310+0000 7fba9bb1c640 1 -- 172.21.15.71:0/2470761177 wait complete. 2024-04-16T15:41:02.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.311+0000 7fba9bb1c640 1 Processor -- start 2024-04-16T15:41:02.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.311+0000 7fba9bb1c640 1 -- start start 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9bb1c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940b0f40 0x7fba94157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9bb1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba940c9ad0 0x7fba941ef940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9bb1c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba94156820 0x7fba94150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9bb1c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fba940c95c0 con 0x7fba94156820 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9bb1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba9406dcb0 con 0x7fba940c9ad0 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9bb1c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fba940c9740 con 0x7fba940b0f40 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9a0b9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba94156820 0x7fba94150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9a0b9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba94156820 0x7fba94150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55012/0 (socket says 172.21.15.71:55012) 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9a0b9640 1 -- 172.21.15.71:0/1937902087 learned_addr learned my addr 172.21.15.71:0/1937902087 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba990b7640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba940c9ad0 0x7fba941ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba998b8640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940b0f40 0x7fba94157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9a0b9640 1 -- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940b0f40 msgr2=0x7fba94157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9a0b9640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940b0f40 0x7fba94157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9a0b9640 1 -- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba940c9ad0 msgr2=0x7fba941ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9a0b9640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba940c9ad0 0x7fba941ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba9a0b9640 1 -- 172.21.15.71:0/1937902087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba941eb9c0 con 0x7fba94156820 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba998b8640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940b0f40 0x7fba94157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.312+0000 7fba990b7640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba940c9ad0 0x7fba941ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.313+0000 7fba9a0b9640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba94156820 0x7fba94150660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c820 tx=0x7fba8c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.313+0000 7fba8a7fc640 1 -- 172.21.15.71:0/1937902087 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c015070 con 0x7fba94156820 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.313+0000 7fba8a7fc640 1 -- 172.21.15.71:0/1937902087 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8c004020 con 0x7fba94156820 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.313+0000 7fba8a7fc640 1 -- 172.21.15.71:0/1937902087 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c004410 con 0x7fba94156820 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.313+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba94155980 con 0x7fba94156820 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.313+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba94155f40 con 0x7fba94156820 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.314+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba940cc4e0 con 0x7fba94156820 2024-04-16T15:41:02.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.314+0000 7fba8a7fc640 1 -- 172.21.15.71:0/1937902087 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba8c007830 con 0x7fba94156820 2024-04-16T15:41:02.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.314+0000 7fba8a7fc640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.315+0000 7fba8a7fc640 1 -- 172.21.15.71:0/1937902087 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fba8c08bbe0 con 0x7fba94156820 2024-04-16T15:41:02.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.315+0000 7fba998b8640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.315+0000 7fba998b8640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fba7806c8f0 0x7fba7806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba90035000 tx=0x7fba90030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.319+0000 7fba8a7fc640 1 -- 172.21.15.71:0/1937902087 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba940cc4e0 con 0x7fba94156820 2024-04-16T15:41:02.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.493+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fba9406e4c0 con 0x7fba94156820 2024-04-16T15:41:02.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.493+0000 7fba8a7fc640 1 -- 172.21.15.71:0/1937902087 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fba8c059d60 con 0x7fba94156820 2024-04-16T15:41:02.492 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:02.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fba7806c8f0 msgr2=0x7fba7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fba7806c8f0 0x7fba7806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba90035000 tx=0x7fba90030000 comp rx=0 tx=0).stop 2024-04-16T15:41:02.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba94156820 msgr2=0x7fba94150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba94156820 0x7fba94150660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c820 tx=0x7fba8c00cce0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 shutdown_connections 2024-04-16T15:41:02.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fba7806c8f0 0x7fba7806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba94156820 0x7fba94150660 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba940c9ad0 0x7fba941ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 --2- 172.21.15.71:0/1937902087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba940b0f40 0x7fba94157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 >> 172.21.15.71:0/1937902087 conn(0x7fba940ad3f0 msgr2=0x7fba9406cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:02.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 shutdown_connections 2024-04-16T15:41:02.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.495+0000 7fba9bb1c640 1 -- 172.21.15.71:0/1937902087 wait complete. 2024-04-16T15:41:02.505 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop' 2024-04-16T15:41:02.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.609+0000 7fc6156c5640 1 Processor -- start 2024-04-16T15:41:02.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.609+0000 7fc6156c5640 1 -- start start 2024-04-16T15:41:02.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc6156c5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc6156c5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc6100ca270 0x7fc6100ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc6156c5640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6100ae4b0 con 0x7fc6100c78d0 2024-04-16T15:41:02.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc6156c5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc6100c9540 con 0x7fc6100ca270 2024-04-16T15:41:02.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc6156c5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc6100c6940 con 0x7fc610159520 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc6067fc640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fc6100c78d0 0x7fc61006df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33572/0 (socket says 172.21.15.71:33572) 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc6067fc640 1 -- 172.21.15.71:0/291120508 learned_addr learned my addr 172.21.15.71:0/291120508 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60f7fe640 1 --2- 172.21.15.71:0/291120508 >> v2:172.21.15.122:3300/0 conn(0x7fc6100ca270 0x7fc6100ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60effd640 1 --2- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60e7fc640 1 -- 172.21.15.71:0/291120508 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445262303 0 0) 0x7fc6100ae4b0 con 0x7fc6100c78d0 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60e7fc640 1 -- 172.21.15.71:0/291120508 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5fc003520 con 0x7fc6100c78d0 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60f7fe640 1 -- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 msgr2=0x7fc610159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60f7fe640 1 --2- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60f7fe640 1 -- 172.21.15.71:0/291120508 >> v1:172.21.15.71:6789/0 conn(0x7fc6100c78d0 legacy=0x7fc61006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60f7fe640 1 -- 172.21.15.71:0/291120508 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61015a030 con 0x7fc6100ca270 2024-04-16T15:41:02.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.610+0000 7fc60effd640 1 --2- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.611+0000 7fc60f7fe640 1 --2- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 0x7fc6100ca700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc608009d90 tx=0x7fc60802bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72990d9572332b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.611+0000 7fc60e7fc640 1 -- 172.21.15.71:0/291120508 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608039070 con 0x7fc6100ca270 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.611+0000 7fc60e7fc640 1 -- 172.21.15.71:0/291120508 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60802cb30 con 0x7fc6100ca270 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.611+0000 7fc60e7fc640 1 -- 172.21.15.71:0/291120508 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608034440 con 0x7fc6100ca270 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.612+0000 7fc6156c5640 1 -- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 msgr2=0x7fc6100ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.612+0000 7fc6156c5640 1 --2- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 0x7fc6100ca700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc608009d90 tx=0x7fc60802bd80 comp rx=0 tx=0).stop 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.612+0000 7fc6156c5640 1 -- 172.21.15.71:0/291120508 shutdown_connections 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.612+0000 7fc6156c5640 1 --2- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 0x7fc6100ca700 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.612+0000 7fc6156c5640 1 --2- 172.21.15.71:0/291120508 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.612+0000 7fc6156c5640 1 -- 172.21.15.71:0/291120508 >> 172.21.15.71:0/291120508 conn(0x7fc6100cb820 msgr2=0x7fc6100cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.612+0000 7fc6156c5640 1 -- 172.21.15.71:0/291120508 shutdown_connections 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.612+0000 7fc6156c5640 1 -- 172.21.15.71:0/291120508 wait complete. 2024-04-16T15:41:02.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.613+0000 7fc6156c5640 1 Processor -- start 2024-04-16T15:41:02.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.613+0000 7fc6156c5640 1 -- start start 2024-04-16T15:41:02.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc6156c5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6100c78d0 0x7fc6100c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc6156c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 0x7fc6101502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc6156c5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc6156c5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc6100c9d50 con 0x7fc6100c78d0 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc6156c5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6100c9a70 con 0x7fc6100ca270 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc6156c5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc6100cac40 con 0x7fc610159520 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6100c78d0 0x7fc6100c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6100c78d0 0x7fc6100c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55014/0 (socket says 172.21.15.71:55014) 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60effd640 1 -- 172.21.15.71:0/3578668203 learned_addr learned my addr 172.21.15.71:0/3578668203 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc6067fc640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 0x7fc6101502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60f7fe640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60effd640 1 -- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 msgr2=0x7fc610149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60effd640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60effd640 1 -- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 msgr2=0x7fc6101502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60effd640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 0x7fc6101502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60effd640 1 -- 172.21.15.71:0/3578668203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc610154080 con 0x7fc6100c78d0 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc6067fc640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 0x7fc6101502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.614+0000 7fc60f7fe640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:02.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.615+0000 7fc60effd640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6100c78d0 0x7fc6100c2710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc60000e960 tx=0x7fc60000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.615+0000 7fc607fff640 1 -- 172.21.15.71:0/3578668203 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600017070 con 0x7fc6100c78d0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.615+0000 7fc607fff640 1 -- 172.21.15.71:0/3578668203 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60000c9a0 con 0x7fc6100c78d0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.615+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6101ef4c0 con 0x7fc6100c78d0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.615+0000 7fc607fff640 1 -- 172.21.15.71:0/3578668203 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60000cd70 con 0x7fc6100c78d0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.615+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6101ef7a0 con 0x7fc6100c78d0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.616+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc610074060 con 0x7fc6100c78d0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.616+0000 7fc607fff640 1 -- 172.21.15.71:0/3578668203 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc60001a070 con 0x7fc6100c78d0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.617+0000 7fc607fff640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc5f006c8f0 0x7fc5f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.617+0000 7fc607fff640 1 -- 172.21.15.71:0/3578668203 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc60008bc70 con 0x7fc6100c78d0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.617+0000 7fc6067fc640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc5f006c8f0 0x7fc5f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:02.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.617+0000 7fc6067fc640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc5f006c8f0 0x7fc5f006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8007fd0 tx=0x7fc5f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:02.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.621+0000 7fc607fff640 1 -- 172.21.15.71:0/3578668203 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc600059df0 con 0x7fc6100c78d0 2024-04-16T15:41:02.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:02.863+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]} v 0) -- 0x7fc6100b0ca0 con 0x7fc6100c78d0 2024-04-16T15:41:03.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.857+0000 7fc607fff640 1 -- 172.21.15.71:0/3578668203 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]}]=0 v62) ==== 146+0+0 (secure 0 0 0) 0x7fc6100b0ca0 con 0x7fc6100c78d0 2024-04-16T15:41:03.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.861+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc5f006c8f0 msgr2=0x7fc5f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:03.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.861+0000 7fc6156c5640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc5f006c8f0 0x7fc5f006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8007fd0 tx=0x7fc5f8009040 comp rx=0 tx=0).stop 2024-04-16T15:41:03.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.861+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6100c78d0 msgr2=0x7fc6100c2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:03.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.861+0000 7fc6156c5640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6100c78d0 0x7fc6100c2710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc60000e960 tx=0x7fc60000ee20 comp rx=0 tx=0).stop 2024-04-16T15:41:03.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.862+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 shutdown_connections 2024-04-16T15:41:03.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.862+0000 7fc6156c5640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc5f006c8f0 0x7fc5f006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.862+0000 7fc6156c5640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc610159520 0x7fc610149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.862+0000 7fc6156c5640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6100ca270 0x7fc6101502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.862+0000 7fc6156c5640 1 --2- 172.21.15.71:0/3578668203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6100c78d0 0x7fc6100c2710 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.862+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 >> 172.21.15.71:0/3578668203 conn(0x7fc6100cb820 msgr2=0x7fc6100ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:03.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.863+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 shutdown_connections 2024-04-16T15:41:03.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.863+0000 7fc6156c5640 1 -- 172.21.15.71:0/3578668203 wait complete. 2024-04-16T15:41:03.880 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-16T15:41:03.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.992+0000 7f43290e4640 1 Processor -- start 2024-04-16T15:41:03.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.992+0000 7f43290e4640 1 -- start start 2024-04-16T15:41:03.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f43290e4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f43240c8c10 0x7f43240c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:03.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f43290e4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43241564f0 0x7f4324071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:03.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f43290e4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4324155d30 con 0x7f4324153320 2024-04-16T15:41:03.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f43290e4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f43240af4c0 con 0x7f43240c8c10 2024-04-16T15:41:03.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f43290e4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f43240c3d50 con 0x7f43241564f0 2024-04-16T15:41:03.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f432359e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43241564f0 0x7f4324071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f431a59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f43240c8c10 0x7f43240c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f4322d9d640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f4324153320 0x7f43240c4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:54256/0 (socket says 172.21.15.71:54256) 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f431a59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f43240c8c10 0x7f43240c26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:54162/0 (socket says 172.21.15.71:54162) 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f432359e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43241564f0 0x7f4324071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:55428/0 (socket says 172.21.15.71:55428) 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f4322d9d640 1 -- 172.21.15.71:0/3204230131 learned_addr learned my addr 172.21.15.71:0/3204230131 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f431a59c640 1 -- 172.21.15.71:0/3204230131 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43241564f0 msgr2=0x7f4324071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f431a59c640 1 --2- 172.21.15.71:0/3204230131 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43241564f0 0x7f4324071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f431a59c640 1 -- 172.21.15.71:0/3204230131 >> v1:172.21.15.122:6789/0 conn(0x7f4324153320 legacy=0x7f43240c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f431a59c640 1 -- 172.21.15.71:0/3204230131 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43240c4030 con 0x7f43240c8c10 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f432259c640 1 -- 172.21.15.71:0/3204230131 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4025677142 0 0) 0x7f4324155d30 con 0x7f4324153320 2024-04-16T15:41:03.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.993+0000 7f432359e640 1 --2- 172.21.15.71:0/3204230131 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43241564f0 0x7f4324071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.994+0000 7f431a59c640 1 --2- 172.21.15.71:0/3204230131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 0x7f43240c26e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f430c0097a0 tx=0x7f430c02b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82abc4712b2eed3e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.994+0000 7f432259c640 1 -- 172.21.15.71:0/3204230131 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c038070 con 0x7f43240c8c10 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.994+0000 7f432259c640 1 -- 172.21.15.71:0/3204230131 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c031700 con 0x7f43240c8c10 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.994+0000 7f432259c640 1 -- 172.21.15.71:0/3204230131 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c031ad0 con 0x7f43240c8c10 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.995+0000 7f43290e4640 1 -- 172.21.15.71:0/3204230131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 msgr2=0x7f43240c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.995+0000 7f43290e4640 1 --2- 172.21.15.71:0/3204230131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 0x7f43240c26e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f430c0097a0 tx=0x7f430c02b7a0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.995+0000 7f43290e4640 1 -- 172.21.15.71:0/3204230131 shutdown_connections 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.995+0000 7f43290e4640 1 --2- 172.21.15.71:0/3204230131 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43241564f0 0x7f4324071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.995+0000 7f43290e4640 1 --2- 172.21.15.71:0/3204230131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 0x7f43240c26e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.995+0000 7f43290e4640 1 -- 172.21.15.71:0/3204230131 >> 172.21.15.71:0/3204230131 conn(0x7f43240c5db0 msgr2=0x7f43240c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:03.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.995+0000 7f43290e4640 1 -- 172.21.15.71:0/3204230131 shutdown_connections 2024-04-16T15:41:03.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.995+0000 7f43290e4640 1 -- 172.21.15.71:0/3204230131 wait complete. 2024-04-16T15:41:03.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.996+0000 7f43290e4640 1 Processor -- start 2024-04-16T15:41:03.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.996+0000 7f43290e4640 1 -- start start 2024-04-16T15:41:03.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f43290e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 0x7f432414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:03.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f43290e4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4324153320 0x7f432414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:03.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f43290e4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43241564f0 0x7f432414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:03.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f43290e4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4324156ac0 con 0x7f43241564f0 2024-04-16T15:41:03.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f43290e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4324156940 con 0x7f43240c8c10 2024-04-16T15:41:03.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f43290e4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4324156c40 con 0x7f4324153320 2024-04-16T15:41:03.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f432359e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43241564f0 0x7f432414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:03.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f432359e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43241564f0 0x7f432414f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55018/0 (socket says 172.21.15.71:55018) 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f432359e640 1 -- 172.21.15.71:0/4268460038 learned_addr learned my addr 172.21.15.71:0/4268460038 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f4322d9d640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 0x7f432414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.997+0000 7f431a59c640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4324153320 0x7f432414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f432359e640 1 -- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4324153320 msgr2=0x7f432414f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f432359e640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4324153320 0x7f432414f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f432359e640 1 -- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 msgr2=0x7f432414a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f432359e640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 0x7f432414a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f432359e640 1 -- 172.21.15.71:0/4268460038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432414ee40 con 0x7f43241564f0 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f431a59c640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4324153320 0x7f432414f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f432359e640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43241564f0 0x7f432414f860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f431c00efc0 tx=0x7f431c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f431bfff640 1 -- 172.21.15.71:0/4268460038 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c018030 con 0x7f43241564f0 2024-04-16T15:41:03.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f431bfff640 1 -- 172.21.15.71:0/4268460038 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431c00cdf0 con 0x7f43241564f0 2024-04-16T15:41:04.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4324157260 con 0x7f43241564f0 2024-04-16T15:41:04.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f431bfff640 1 -- 172.21.15.71:0/4268460038 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c003bb0 con 0x7f43241564f0 2024-04-16T15:41:04.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.998+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43241ecae0 con 0x7f43241564f0 2024-04-16T15:41:04.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:03.999+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432414f010 con 0x7f43241564f0 2024-04-16T15:41:04.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.000+0000 7f431bfff640 1 -- 172.21.15.71:0/4268460038 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f431c005d10 con 0x7f43241564f0 2024-04-16T15:41:04.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.000+0000 7f431bfff640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f430406c8f0 0x7f430406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.000+0000 7f431bfff640 1 -- 172.21.15.71:0/4268460038 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6764+0+0 (secure 0 0 0) 0x7f431c08c890 con 0x7f43241564f0 2024-04-16T15:41:04.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.000+0000 7f4322d9d640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f430406c8f0 0x7f430406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.000+0000 7f4322d9d640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f430406c8f0 0x7f430406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4314000f60 tx=0x7f4314007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:04.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.004+0000 7f431bfff640 1 -- 172.21.15.71:0/4268460038 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f431c022070 con 0x7f43241564f0 2024-04-16T15:41:04.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.180+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4324068be0 con 0x7f43241564f0 2024-04-16T15:41:04.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.183+0000 7f431bfff640 1 -- 172.21.15.71:0/4268460038 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7f431c010040 con 0x7f43241564f0 2024-04-16T15:41:04.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f430406c8f0 msgr2=0x7f430406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f430406c8f0 0x7f430406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4314000f60 tx=0x7f4314007d50 comp rx=0 tx=0).stop 2024-04-16T15:41:04.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43241564f0 msgr2=0x7f432414f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43241564f0 0x7f432414f860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f431c00efc0 tx=0x7f431c00c490 comp rx=0 tx=0).stop 2024-04-16T15:41:04.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 shutdown_connections 2024-04-16T15:41:04.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f430406c8f0 0x7f430406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43241564f0 0x7f432414f860 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4324153320 0x7f432414f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 --2- 172.21.15.71:0/4268460038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43240c8c10 0x7f432414a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 >> 172.21.15.71:0/4268460038 conn(0x7f43240c5db0 msgr2=0x7f4324153a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:04.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 shutdown_connections 2024-04-16T15:41:04.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.185+0000 7f43290e4640 1 -- 172.21.15.71:0/4268460038 wait complete. 2024-04-16T15:41:04.195 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-16T15:41:04.195 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-16T15:41:04.246 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-16T15:41:04.303 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-16T15:41:04.303 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:41:04.303 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:41:04.320 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:41:04.320 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:41:04.376 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:41:04.377 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:41:04.433 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:04.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.578+0000 7ff06af32640 1 Processor -- start 2024-04-16T15:41:04.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.578+0000 7ff06af32640 1 -- start start 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff06af32640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c2990 0x7ff06415f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff06af32640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff064153ba0 con 0x7ff0640ab950 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff06af32640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff064153900 con 0x7ff064150f40 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff06af32640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff0640ad3e0 con 0x7ff0640c2990 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff063fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7ff0640ab950 0x7ff0640aa3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33580/0 (socket says 172.21.15.71:33580) 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff063fff640 1 -- 172.21.15.71:0/587804488 learned_addr learned my addr 172.21.15.71:0/587804488 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785020364 0 0) 0x7ff064153ba0 con 0x7ff0640ab950 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff054003520 con 0x7ff0640ab950 2024-04-16T15:41:04.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0694cf640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c2990 0x7ff06415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 714956330 0 0) 0x7ff054003520 con 0x7ff0640ab950 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c2990 msgr2=0x7ff06415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c2990 0x7ff06415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 >> v1:172.21.15.122:6789/0 conn(0x7ff064150f40 legacy=0x7ff0640accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0640ac4d0 con 0x7ff0640ab950 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7ff050001690 con 0x7ff0640ab950 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 >> v1:172.21.15.71:6789/0 conn(0x7ff0640ab950 legacy=0x7ff0640aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c2990 0x7ff0540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff054003f60 0x7ff054004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054004d40 0x7ff0540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640ad3e0 con 0x7ff054003f60 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff05400bb30 con 0x7ff054004d40 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff05400bd30 con 0x7ff0640c2990 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.579+0000 7ff063fff640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff054003f60 0x7ff054004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.580+0000 7ff068cce640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c2990 0x7ff0540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.580+0000 7ff0694cf640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054004d40 0x7ff0540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.580+0000 7ff063fff640 1 -- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c2990 msgr2=0x7ff0540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.580+0000 7ff063fff640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c2990 0x7ff0540039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.580+0000 7ff063fff640 1 -- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054004d40 msgr2=0x7ff0540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.580+0000 7ff063fff640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054004d40 0x7ff0540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.580+0000 7ff063fff640 1 -- 172.21.15.71:0/587804488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff054003520 con 0x7ff054003f60 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff063fff640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff054003f60 0x7ff054004800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c210 tx=0x7ff054005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a4e4b700b41cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050007d10 con 0x7ff054003f60 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff050030070 con 0x7ff054003f60 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff0637fe640 1 -- 172.21.15.71:0/587804488 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050005d70 con 0x7ff054003f60 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff06af32640 1 -- 172.21.15.71:0/587804488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff054003f60 msgr2=0x7ff054004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff06af32640 1 --2- 172.21.15.71:0/587804488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff054003f60 0x7ff054004800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c210 tx=0x7ff054005830 comp rx=0 tx=0).stop 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff0694cf640 1 -- 172.21.15.71:0/587804488 reap_dead start 2024-04-16T15:41:04.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff06af32640 1 -- 172.21.15.71:0/587804488 shutdown_connections 2024-04-16T15:41:04.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff06af32640 1 -- 172.21.15.71:0/587804488 >> 172.21.15.71:0/587804488 conn(0x7ff0640c3e10 msgr2=0x7ff0640c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:04.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff06af32640 1 -- 172.21.15.71:0/587804488 shutdown_connections 2024-04-16T15:41:04.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.581+0000 7ff06af32640 1 -- 172.21.15.71:0/587804488 wait complete. 2024-04-16T15:41:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.582+0000 7ff06af32640 1 Processor -- start 2024-04-16T15:41:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.582+0000 7ff06af32640 1 -- start start 2024-04-16T15:41:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff06af32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054003f60 0x7ff064149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff06af32640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c1740 0x7ff064151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff06af32640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0640c4f50 0x7ff064157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff06af32640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640aaad0 con 0x7ff0640c4f50 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff06af32640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640c1f20 con 0x7ff054003f60 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff06af32640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff0640aac50 con 0x7ff0640c1740 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff0694cf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0640c4f50 0x7ff064157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff0694cf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0640c4f50 0x7ff064157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55036/0 (socket says 172.21.15.71:55036) 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff0694cf640 1 -- 172.21.15.71:0/496644257 learned_addr learned my addr 172.21.15.71:0/496644257 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff068cce640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054003f60 0x7ff064149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff063fff640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c1740 0x7ff064151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff0694cf640 1 -- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c1740 msgr2=0x7ff064151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff0694cf640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c1740 0x7ff064151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff0694cf640 1 -- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054003f60 msgr2=0x7ff064149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff0694cf640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054003f60 0x7ff064149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff0694cf640 1 -- 172.21.15.71:0/496644257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06414bf50 con 0x7ff0640c4f50 2024-04-16T15:41:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.583+0000 7ff068cce640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054003f60 0x7ff064149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.584+0000 7ff063fff640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c1740 0x7ff064151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.584+0000 7ff0694cf640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0640c4f50 0x7ff064157280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff064157dc0 tx=0x7ff05c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.584+0000 7ff0617fa640 1 -- 172.21.15.71:0/496644257 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00fde0 con 0x7ff0640c4f50 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.584+0000 7ff0617fa640 1 -- 172.21.15.71:0/496644257 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff05c01d070 con 0x7ff0640c4f50 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.584+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0641ed500 con 0x7ff0640c4f50 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.584+0000 7ff0617fa640 1 -- 172.21.15.71:0/496644257 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c015670 con 0x7ff0640c4f50 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.584+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff064151ba0 con 0x7ff0640c4f50 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.585+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff064068d20 con 0x7ff0640c4f50 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.585+0000 7ff0617fa640 1 -- 172.21.15.71:0/496644257 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff05c015810 con 0x7ff0640c4f50 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.586+0000 7ff0617fa640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff04806c8f0 0x7ff04806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.586+0000 7ff0617fa640 1 -- 172.21.15.71:0/496644257 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6764+0+0 (secure 0 0 0) 0x7ff05c08c970 con 0x7ff0640c4f50 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.586+0000 7ff068cce640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff04806c8f0 0x7ff04806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.586+0000 7ff068cce640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff04806c8f0 0x7ff04806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0580035f0 tx=0x7ff058009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:04.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.590+0000 7ff0617fa640 1 -- 172.21.15.71:0/496644257 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff05c010040 con 0x7ff0640c4f50 2024-04-16T15:41:04.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.768+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff06406d680 con 0x7ff0640c4f50 2024-04-16T15:41:04.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.769+0000 7ff0617fa640 1 -- 172.21.15.71:0/496644257 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18938 (secure 0 0 0) 0x7ff05c05a8f0 con 0x7ff0640c4f50 2024-04-16T15:41:04.768 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:04.768 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:03.851367+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:04.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.771+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff04806c8f0 msgr2=0x7ff04806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.771+0000 7ff06af32640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff04806c8f0 0x7ff04806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0580035f0 tx=0x7ff058009040 comp rx=0 tx=0).stop 2024-04-16T15:41:04.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.771+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0640c4f50 msgr2=0x7ff064157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.771+0000 7ff06af32640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0640c4f50 0x7ff064157280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff064157dc0 tx=0x7ff05c00f580 comp rx=0 tx=0).stop 2024-04-16T15:41:04.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.772+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 shutdown_connections 2024-04-16T15:41:04.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.772+0000 7ff06af32640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff04806c8f0 0x7ff04806edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.772+0000 7ff06af32640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0640c4f50 0x7ff064157280 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.772+0000 7ff06af32640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0640c1740 0x7ff064151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.772+0000 7ff06af32640 1 --2- 172.21.15.71:0/496644257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff054003f60 0x7ff064149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.772+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 >> 172.21.15.71:0/496644257 conn(0x7ff0640c3e10 msgr2=0x7ff06414eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:04.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.772+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 shutdown_connections 2024-04-16T15:41:04.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.772+0000 7ff06af32640 1 -- 172.21.15.71:0/496644257 wait complete. 2024-04-16T15:41:04.782 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:41:04.782 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:41:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.896+0000 7f2fa46c7640 1 Processor -- start 2024-04-16T15:41:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.896+0000 7f2fa46c7640 1 -- start start 2024-04-16T15:41:04.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.897+0000 7f2fa46c7640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2f9c14ffe0 0x7f2f9c155e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.897+0000 7f2fa46c7640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2f9c06cec0 0x7f2f9c156380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.897+0000 7f2fa46c7640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f9c06cc60 con 0x7f2f9c153d40 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.897+0000 7f2fa46c7640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f2f9c0c2720 con 0x7f2f9c06cec0 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.897+0000 7f2fa46c7640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2f9c0c2fc0 con 0x7f2f9c14ffe0 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.898+0000 7f2fa1c62640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2f9c06cec0 0x7f2f9c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.898+0000 7f2fa1c62640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2f9c06cec0 0x7f2f9c156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55044/0 (socket says 172.21.15.71:55044) 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.898+0000 7f2fa1c62640 1 -- 172.21.15.71:0/2618582246 learned_addr learned my addr 172.21.15.71:0/2618582246 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.898+0000 7f2fa2463640 1 --2- 172.21.15.71:0/2618582246 >> v2:172.21.15.122:3300/0 conn(0x7f2f9c14ffe0 0x7f2f9c155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.898+0000 7f2fa1c62640 1 -- 172.21.15.71:0/2618582246 >> v2:172.21.15.122:3300/0 conn(0x7f2f9c14ffe0 msgr2=0x7f2f9c155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.898+0000 7f2fa1c62640 1 --2- 172.21.15.71:0/2618582246 >> v2:172.21.15.122:3300/0 conn(0x7f2f9c14ffe0 0x7f2f9c155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.898+0000 7f2fa1c62640 1 -- 172.21.15.71:0/2618582246 >> v1:172.21.15.122:6789/0 conn(0x7f2f9c153d40 legacy=0x7f2f9c1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.898+0000 7f2fa1c62640 1 -- 172.21.15.71:0/2618582246 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c1569f0 con 0x7f2f9c06cec0 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa1c62640 1 --2- 172.21.15.71:0/2618582246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c06cec0 0x7f2f9c156380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0c7e70 tx=0x7f2f8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f28e81e10d34f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa1461640 1 -- 172.21.15.71:0/2618582246 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c02e070 con 0x7f2f9c06cec0 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa1461640 1 -- 172.21.15.71:0/2618582246 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8c037070 con 0x7f2f9c06cec0 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa1461640 1 -- 172.21.15.71:0/2618582246 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c031a00 con 0x7f2f9c06cec0 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa46c7640 1 -- 172.21.15.71:0/2618582246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c06cec0 msgr2=0x7f2f9c156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/2618582246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c06cec0 0x7f2f9c156380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0c7e70 tx=0x7f2f8c02b880 comp rx=0 tx=0).stop 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa46c7640 1 -- 172.21.15.71:0/2618582246 shutdown_connections 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/2618582246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c06cec0 0x7f2f9c156380 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/2618582246 >> v2:172.21.15.122:3300/0 conn(0x7f2f9c14ffe0 0x7f2f9c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa46c7640 1 -- 172.21.15.71:0/2618582246 >> 172.21.15.71:0/2618582246 conn(0x7f2f9c0c7940 msgr2=0x7f2f9c0c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa46c7640 1 -- 172.21.15.71:0/2618582246 shutdown_connections 2024-04-16T15:41:04.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.899+0000 7f2fa46c7640 1 -- 172.21.15.71:0/2618582246 wait complete. 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa46c7640 1 Processor -- start 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa46c7640 1 -- start start 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa46c7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f9c06cec0 0x7f2f9c06c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa46c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f9c14ffe0 0x7f2f9c150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa46c7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c153d40 0x7f2f9c1514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa46c7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c0c4740 con 0x7f2f9c153d40 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa46c7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c0c2c50 con 0x7f2f9c14ffe0 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa46c7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9c0c3450 con 0x7f2f9c06cec0 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa2c64640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c153d40 0x7f2f9c1514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa2c64640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c153d40 0x7f2f9c1514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55060/0 (socket says 172.21.15.71:55060) 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.901+0000 7f2fa2c64640 1 -- 172.21.15.71:0/272667164 learned_addr learned my addr 172.21.15.71:0/272667164 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa1c62640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f9c14ffe0 0x7f2f9c150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa2463640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f9c06cec0 0x7f2f9c06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa2c64640 1 -- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f9c06cec0 msgr2=0x7f2f9c06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa2c64640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f9c06cec0 0x7f2f9c06c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa2c64640 1 -- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f9c14ffe0 msgr2=0x7f2f9c150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa2c64640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f9c14ffe0 0x7f2f9c150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa2c64640 1 -- 172.21.15.71:0/272667164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c149140 con 0x7f2f9c153d40 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa2463640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f9c06cec0 0x7f2f9c06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa2c64640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c153d40 0x7f2f9c1514e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c760 tx=0x7f2f9400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2f92ffd640 1 -- 172.21.15.71:0/272667164 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94003b70 con 0x7f2f9c153d40 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2f92ffd640 1 -- 172.21.15.71:0/272667164 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f94003d10 con 0x7f2f9c153d40 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9c0ab320 con 0x7f2f9c153d40 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2f92ffd640 1 -- 172.21.15.71:0/272667164 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400aa90 con 0x7f2f9c153d40 2024-04-16T15:41:04.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.902+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9c0ab8e0 con 0x7f2f9c153d40 2024-04-16T15:41:04.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.903+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9c068be0 con 0x7f2f9c153d40 2024-04-16T15:41:04.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.903+0000 7f2f92ffd640 1 -- 172.21.15.71:0/272667164 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f9400ac30 con 0x7f2f9c153d40 2024-04-16T15:41:04.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.904+0000 7f2f92ffd640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2f8006c8f0 0x7f2f8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:04.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.904+0000 7f2f92ffd640 1 -- 172.21.15.71:0/272667164 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f2f9408be30 con 0x7f2f9c153d40 2024-04-16T15:41:04.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.904+0000 7f2fa2463640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2f8006c8f0 0x7f2f8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:04.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.904+0000 7f2fa2463640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2f8006c8f0 0x7f2f8006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009ac0 tx=0x7f2f98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:04.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:04.908+0000 7f2f92ffd640 1 -- 172.21.15.71:0/272667164 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f94059ed0 con 0x7f2f9c153d40 2024-04-16T15:41:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.083+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2f9c0c4520 con 0x7f2f9c153d40 2024-04-16T15:41:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.084+0000 7f2f92ffd640 1 -- 172.21.15.71:0/272667164 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2f940597d0 con 0x7f2f9c153d40 2024-04-16T15:41:05.083 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:41:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.085+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2f8006c8f0 msgr2=0x7f2f8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.085+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2f8006c8f0 0x7f2f8006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009ac0 tx=0x7f2f98008040 comp rx=0 tx=0).stop 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c153d40 msgr2=0x7f2f9c1514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c153d40 0x7f2f9c1514e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c760 tx=0x7f2f9400cc20 comp rx=0 tx=0).stop 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 shutdown_connections 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2f8006c8f0 0x7f2f8006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f9c153d40 0x7f2f9c1514e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f9c14ffe0 0x7f2f9c150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 --2- 172.21.15.71:0/272667164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f9c06cec0 0x7f2f9c06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 >> 172.21.15.71:0/272667164 conn(0x7f2f9c0c7940 msgr2=0x7f2f9c1541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 shutdown_connections 2024-04-16T15:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.086+0000 7f2fa46c7640 1 -- 172.21.15.71:0/272667164 wait complete. 2024-04-16T15:41:05.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:41:05.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.203+0000 7fa91e19d640 1 Processor -- start 2024-04-16T15:41:05.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.203+0000 7fa91e19d640 1 -- start start 2024-04-16T15:41:05.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa91e19d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa9180c94c0 0x7fa9181564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa91e19d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa91806e820 con 0x7fa918153450 2024-04-16T15:41:05.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa91e19d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa918159fb0 con 0x7fa9180c99d0 2024-04-16T15:41:05.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa91e19d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fa9180c8150 con 0x7fa9180c94c0 2024-04-16T15:41:05.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9177fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fa918153450 0x7fa9180c5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33592/0 (socket says 172.21.15.71:33592) 2024-04-16T15:41:05.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9177fe640 1 -- 172.21.15.71:0/1110626603 learned_addr learned my addr 172.21.15.71:0/1110626603 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:05.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3455509019 0 0) 0x7fa91806e820 con 0x7fa918153450 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa900003520 con 0x7fa918153450 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa917fff640 1 --2- 172.21.15.71:0/1110626603 >> v2:172.21.15.122:3300/0 conn(0x7fa9180c94c0 0x7fa9181564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1874401278 0 0) 0x7fa900003520 con 0x7fa918153450 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 >> v2:172.21.15.122:3300/0 conn(0x7fa9180c94c0 msgr2=0x7fa9181564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 --2- 172.21.15.71:0/1110626603 >> v2:172.21.15.122:3300/0 conn(0x7fa9180c94c0 0x7fa9181564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 >> v1:172.21.15.122:6789/0 conn(0x7fa9180c99d0 legacy=0x7fa918155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9180c8430 con 0x7fa918153450 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fa908001670 con 0x7fa918153450 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 >> v1:172.21.15.71:6789/0 conn(0x7fa918153450 legacy=0x7fa9180c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9000039b0 0x7fa900003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa900004800 0x7fa900004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.204+0000 7fa9167fc640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9000052e0 0x7fa90000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa9180c8150 con 0x7fa900004800 2024-04-16T15:41:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa90000c1b0 con 0x7fa9000039b0 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa90000c3b0 con 0x7fa9000052e0 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa916ffd640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa900004800 0x7fa900004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa9177fe640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9000039b0 0x7fa900003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa917fff640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9000052e0 0x7fa90000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa916ffd640 1 -- 172.21.15.71:0/1110626603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9000052e0 msgr2=0x7fa90000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa916ffd640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9000052e0 0x7fa90000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa916ffd640 1 -- 172.21.15.71:0/1110626603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9000039b0 msgr2=0x7fa900003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa916ffd640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9000039b0 0x7fa900003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa916ffd640 1 -- 172.21.15.71:0/1110626603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa900003520 con 0x7fa900004800 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.205+0000 7fa917fff640 1 -- 172.21.15.71:0/1110626603 reap_dead start 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa916ffd640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa900004800 0x7fa900004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa91806c9f0 tx=0x7fa900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec77bc119ff42aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c001a70 con 0x7fa900004800 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90c003b70 con 0x7fa900004800 2024-04-16T15:41:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa9167fc640 1 -- 172.21.15.71:0/1110626603 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0050d0 con 0x7fa900004800 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa91e19d640 1 -- 172.21.15.71:0/1110626603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa900004800 msgr2=0x7fa900004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa91e19d640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa900004800 0x7fa900004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa91806c9f0 tx=0x7fa900005830 comp rx=0 tx=0).stop 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa91e19d640 1 -- 172.21.15.71:0/1110626603 shutdown_connections 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa91e19d640 1 --2- 172.21.15.71:0/1110626603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa900004800 0x7fa900004c80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa91e19d640 1 -- 172.21.15.71:0/1110626603 >> 172.21.15.71:0/1110626603 conn(0x7fa9180ad2f0 msgr2=0x7fa9180ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa91e19d640 1 -- 172.21.15.71:0/1110626603 shutdown_connections 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.206+0000 7fa91e19d640 1 -- 172.21.15.71:0/1110626603 wait complete. 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.207+0000 7fa91e19d640 1 Processor -- start 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa91e19d640 1 -- start start 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa91e19d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa900004800 0x7fa918158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa91e19d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9180b0e10 0x7fa9180ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa91e19d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9180aea40 0x7fa918158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa91e19d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa9180c60c0 con 0x7fa9180b0e10 2024-04-16T15:41:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa91e19d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9180c5f40 con 0x7fa900004800 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa91e19d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa9180c6240 con 0x7fa9180aea40 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa916ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9180b0e10 0x7fa9180ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa916ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9180b0e10 0x7fa9180ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55072/0 (socket says 172.21.15.71:55072) 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa916ffd640 1 -- 172.21.15.71:0/3478313089 learned_addr learned my addr 172.21.15.71:0/3478313089 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.208+0000 7fa9177fe640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa900004800 0x7fa918158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa917fff640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9180aea40 0x7fa918158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa916ffd640 1 -- 172.21.15.71:0/3478313089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9180aea40 msgr2=0x7fa918158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa916ffd640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9180aea40 0x7fa918158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa916ffd640 1 -- 172.21.15.71:0/3478313089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa900004800 msgr2=0x7fa918158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa916ffd640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa900004800 0x7fa918158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa916ffd640 1 -- 172.21.15.71:0/3478313089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91814c8f0 con 0x7fa9180b0e10 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa916ffd640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9180b0e10 0x7fa9180ae500 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa90c02bfc0 tx=0x7fa90c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa8fffff640 1 -- 172.21.15.71:0/3478313089 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c006b50 con 0x7fa9180b0e10 2024-04-16T15:41:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa8fffff640 1 -- 172.21.15.71:0/3478313089 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90c03f030 con 0x7fa9180b0e10 2024-04-16T15:41:05.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa918157b50 con 0x7fa9180b0e10 2024-04-16T15:41:05.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa8fffff640 1 -- 172.21.15.71:0/3478313089 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c007030 con 0x7fa9180b0e10 2024-04-16T15:41:05.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.209+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91806de90 con 0x7fa9180b0e10 2024-04-16T15:41:05.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.210+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa918074060 con 0x7fa9180b0e10 2024-04-16T15:41:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.211+0000 7fa8fffff640 1 -- 172.21.15.71:0/3478313089 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa90c0071d0 con 0x7fa9180b0e10 2024-04-16T15:41:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.211+0000 7fa8fffff640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa90406c8f0 0x7fa90406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.211+0000 7fa8fffff640 1 -- 172.21.15.71:0/3478313089 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fa90c0ae1c0 con 0x7fa9180b0e10 2024-04-16T15:41:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.211+0000 7fa9177fe640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa90406c8f0 0x7fa90406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.212+0000 7fa9177fe640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa90406c8f0 0x7fa90406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa9181efa00 tx=0x7fa908000f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.216+0000 7fa8fffff640 1 -- 172.21.15.71:0/3478313089 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa90c07bc10 con 0x7fa9180b0e10 2024-04-16T15:41:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.392+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa91806e4c0 con 0x7fa9180b0e10 2024-04-16T15:41:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.392+0000 7fa8fffff640 1 -- 172.21.15.71:0/3478313089 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fa90c07ff10 con 0x7fa9180b0e10 2024-04-16T15:41:05.392 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa90406c8f0 msgr2=0x7fa90406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa90406c8f0 0x7fa90406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa9181efa00 tx=0x7fa908000f80 comp rx=0 tx=0).stop 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9180b0e10 msgr2=0x7fa9180ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9180b0e10 0x7fa9180ae500 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa90c02bfc0 tx=0x7fa90c0014a0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 shutdown_connections 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa90406c8f0 0x7fa90406edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9180aea40 0x7fa918158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9180b0e10 0x7fa9180ae500 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 --2- 172.21.15.71:0/3478313089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa900004800 0x7fa918158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.394+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 >> 172.21.15.71:0/3478313089 conn(0x7fa9180ad2f0 msgr2=0x7fa91806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.395+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 shutdown_connections 2024-04-16T15:41:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.395+0000 7fa91e19d640 1 -- 172.21.15.71:0/3478313089 wait complete. 2024-04-16T15:41:05.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:41:05.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.512+0000 7f0a9d11c640 1 Processor -- start 2024-04-16T15:41:05.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.512+0000 7f0a9d11c640 1 -- start start 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9d11c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0a980c2920 0x7f0a98154470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9d11c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0a980acbf0 0x7f0a980ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9d11c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a980ac460 con 0x7f0a9814dc00 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9d11c640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f0a980c1e60 con 0x7f0a980acbf0 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9d11c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f0a980c20b0 con 0x7f0a980c2920 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9759e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0a980acbf0 0x7f0a980ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9759e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f0a980acbf0 0x7f0a980ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55086/0 (socket says 172.21.15.71:55086) 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9759e640 1 -- 172.21.15.71:0/63502902 learned_addr learned my addr 172.21.15.71:0/63502902 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9759e640 1 -- 172.21.15.71:0/63502902 >> v2:172.21.15.122:3300/0 conn(0x7f0a980c2920 msgr2=0x7f0a98154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9659c640 1 --2- 172.21.15.71:0/63502902 >> v2:172.21.15.122:3300/0 conn(0x7f0a980c2920 0x7f0a98154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9759e640 1 --2- 172.21.15.71:0/63502902 >> v2:172.21.15.122:3300/0 conn(0x7f0a980c2920 0x7f0a98154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9759e640 1 -- 172.21.15.71:0/63502902 >> v1:172.21.15.122:6789/0 conn(0x7f0a9814dc00 legacy=0x7f0a980aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.513+0000 7f0a9759e640 1 -- 172.21.15.71:0/63502902 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980ac6d0 con 0x7f0a980acbf0 2024-04-16T15:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.514+0000 7f0a9759e640 1 --2- 172.21.15.71:0/63502902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980acbf0 0x7f0a980ad040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a9806ff00 tx=0x7f0a9002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4e5402d7a33933 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.514+0000 7f0a95d9b640 1 -- 172.21.15.71:0/63502902 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90038070 con 0x7f0a980acbf0 2024-04-16T15:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.514+0000 7f0a95d9b640 1 -- 172.21.15.71:0/63502902 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9002f050 con 0x7f0a980acbf0 2024-04-16T15:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.514+0000 7f0a95d9b640 1 -- 172.21.15.71:0/63502902 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90031970 con 0x7f0a980acbf0 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.515+0000 7f0a9d11c640 1 -- 172.21.15.71:0/63502902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980acbf0 msgr2=0x7f0a980ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.515+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/63502902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980acbf0 0x7f0a980ad040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a9806ff00 tx=0x7f0a9002b7f0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.515+0000 7f0a9d11c640 1 -- 172.21.15.71:0/63502902 shutdown_connections 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.515+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/63502902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980acbf0 0x7f0a980ad040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.515+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/63502902 >> v2:172.21.15.122:3300/0 conn(0x7f0a980c2920 0x7f0a98154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.515+0000 7f0a9d11c640 1 -- 172.21.15.71:0/63502902 >> 172.21.15.71:0/63502902 conn(0x7f0a980c7940 msgr2=0x7f0a980c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.515+0000 7f0a9d11c640 1 -- 172.21.15.71:0/63502902 shutdown_connections 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.515+0000 7f0a9d11c640 1 -- 172.21.15.71:0/63502902 wait complete. 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.516+0000 7f0a9d11c640 1 Processor -- start 2024-04-16T15:41:05.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.516+0000 7f0a9d11c640 1 -- start start 2024-04-16T15:41:05.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9d11c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a980acbf0 0x7f0a98148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9d11c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980c2920 0x7f0a98072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9d11c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0a9814dc00 0x7f0a981483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9d11c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980aaad0 con 0x7f0a980c2920 2024-04-16T15:41:05.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9d11c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a980c2390 con 0x7f0a980acbf0 2024-04-16T15:41:05.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9d11c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980aac50 con 0x7f0a9814dc00 2024-04-16T15:41:05.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980c2920 0x7f0a98072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980c2920 0x7f0a98072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55102/0 (socket says 172.21.15.71:55102) 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9659c640 1 -- 172.21.15.71:0/4183304612 learned_addr learned my addr 172.21.15.71:0/4183304612 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a96d9d640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a980acbf0 0x7f0a98148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9759e640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0a9814dc00 0x7f0a981483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9659c640 1 -- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0a9814dc00 msgr2=0x7f0a981483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9659c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0a9814dc00 0x7f0a981483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9659c640 1 -- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a980acbf0 msgr2=0x7f0a98148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9659c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a980acbf0 0x7f0a98148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9659c640 1 -- 172.21.15.71:0/4183304612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a981efa40 con 0x7f0a980c2920 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a96d9d640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a980acbf0 0x7f0a98148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.517+0000 7f0a9759e640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0a9814dc00 0x7f0a981483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.518+0000 7f0a9659c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980c2920 0x7f0a98072d50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c7f0 tx=0x7f0a8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.518+0000 7f0a7f7fe640 1 -- 172.21.15.71:0/4183304612 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c015070 con 0x7f0a980c2920 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.518+0000 7f0a7f7fe640 1 -- 172.21.15.71:0/4183304612 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8c004220 con 0x7f0a980c2920 2024-04-16T15:41:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.518+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a981eec80 con 0x7f0a980c2920 2024-04-16T15:41:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.518+0000 7f0a7f7fe640 1 -- 172.21.15.71:0/4183304612 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c007620 con 0x7f0a980c2920 2024-04-16T15:41:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.518+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a981ef240 con 0x7f0a980c2920 2024-04-16T15:41:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.519+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a98068be0 con 0x7f0a980c2920 2024-04-16T15:41:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.519+0000 7f0a7f7fe640 1 -- 172.21.15.71:0/4183304612 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a8c0077c0 con 0x7f0a980c2920 2024-04-16T15:41:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.520+0000 7f0a7f7fe640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0a8406c8f0 0x7f0a8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.520+0000 7f0a7f7fe640 1 -- 172.21.15.71:0/4183304612 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f0a8c08c0d0 con 0x7f0a980c2920 2024-04-16T15:41:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.520+0000 7f0a96d9d640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0a8406c8f0 0x7f0a8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.520+0000 7f0a96d9d640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0a8406c8f0 0x7f0a8406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007e90 tx=0x7f0a880073a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.524+0000 7f0a7f7fe640 1 -- 172.21.15.71:0/4183304612 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a8c05a170 con 0x7f0a980c2920 2024-04-16T15:41:05.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.698+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a980c4990 con 0x7f0a980c2920 2024-04-16T15:41:05.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.698+0000 7f0a7f7fe640 1 -- 172.21.15.71:0/4183304612 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f0a8c059a70 con 0x7f0a980c2920 2024-04-16T15:41:05.697 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:41:05.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0a8406c8f0 msgr2=0x7f0a8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0a8406c8f0 0x7f0a8406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007e90 tx=0x7f0a880073a0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980c2920 msgr2=0x7f0a98072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980c2920 0x7f0a98072d50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c7f0 tx=0x7f0a8c00ccb0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 shutdown_connections 2024-04-16T15:41:05.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0a8406c8f0 0x7f0a8406edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0a9814dc00 0x7f0a981483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a980c2920 0x7f0a98072d50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 --2- 172.21.15.71:0/4183304612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a980acbf0 0x7f0a98148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 >> 172.21.15.71:0/4183304612 conn(0x7f0a980c7940 msgr2=0x7f0a9806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:05.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 shutdown_connections 2024-04-16T15:41:05.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.700+0000 7f0a9d11c640 1 -- 172.21.15.71:0/4183304612 wait complete. 2024-04-16T15:41:05.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:41:05.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.817+0000 7fe81ef74640 1 Processor -- start 2024-04-16T15:41:05.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.817+0000 7fe81ef74640 1 -- start start 2024-04-16T15:41:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe81ef74640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe8180713a0 0x7fe818155050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe81ef74640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe8180c8ee0 0x7fe818155590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe81ef74640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8181549a0 con 0x7fe8180c8130 2024-04-16T15:41:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe81ef74640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fe8180c7400 con 0x7fe8180713a0 2024-04-16T15:41:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe81ef74640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe8180c7ca0 con 0x7fe8180c8ee0 2024-04-16T15:41:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe81cd10640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fe8180713a0 0x7fe818155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe80ffff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fe8180c8130 0x7fe8180ac5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33604/0 (socket says 172.21.15.71:33604) 2024-04-16T15:41:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe80ffff640 1 -- 172.21.15.71:0/4133276350 learned_addr learned my addr 172.21.15.71:0/4133276350 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.818+0000 7fe81d511640 1 --2- 172.21.15.71:0/4133276350 >> v2:172.21.15.122:3300/0 conn(0x7fe8180c8ee0 0x7fe818155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe81cd10640 1 -- 172.21.15.71:0/4133276350 >> v2:172.21.15.122:3300/0 conn(0x7fe8180c8ee0 msgr2=0x7fe818155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe81cd10640 1 --2- 172.21.15.71:0/4133276350 >> v2:172.21.15.122:3300/0 conn(0x7fe8180c8ee0 0x7fe818155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe81cd10640 1 -- 172.21.15.71:0/4133276350 >> v1:172.21.15.71:6789/0 conn(0x7fe8180c8130 legacy=0x7fe8180ac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe81cd10640 1 -- 172.21.15.71:0/4133276350 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8180c7650 con 0x7fe8180713a0 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe80f7fe640 1 -- 172.21.15.71:0/4133276350 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579823515 0 0) 0x7fe8181549a0 con 0x7fe8180c8130 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe81cd10640 1 --2- 172.21.15.71:0/4133276350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180713a0 0x7fe818155050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe818151750 tx=0x7fe81402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7beb91bbddad1bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe80f7fe640 1 -- 172.21.15.71:0/4133276350 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8140315a0 con 0x7fe8180713a0 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe80f7fe640 1 -- 172.21.15.71:0/4133276350 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe814037070 con 0x7fe8180713a0 2024-04-16T15:41:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.819+0000 7fe80f7fe640 1 -- 172.21.15.71:0/4133276350 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814031960 con 0x7fe8180713a0 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.820+0000 7fe81ef74640 1 -- 172.21.15.71:0/4133276350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180713a0 msgr2=0x7fe818155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.820+0000 7fe81ef74640 1 --2- 172.21.15.71:0/4133276350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180713a0 0x7fe818155050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe818151750 tx=0x7fe81402b840 comp rx=0 tx=0).stop 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.820+0000 7fe81ef74640 1 -- 172.21.15.71:0/4133276350 shutdown_connections 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.820+0000 7fe81ef74640 1 --2- 172.21.15.71:0/4133276350 >> v2:172.21.15.122:3300/0 conn(0x7fe8180c8ee0 0x7fe818155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.820+0000 7fe81ef74640 1 --2- 172.21.15.71:0/4133276350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180713a0 0x7fe818155050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.820+0000 7fe81ef74640 1 -- 172.21.15.71:0/4133276350 >> 172.21.15.71:0/4133276350 conn(0x7fe8180c96e0 msgr2=0x7fe8180c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.820+0000 7fe81ef74640 1 -- 172.21.15.71:0/4133276350 shutdown_connections 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.820+0000 7fe81ef74640 1 -- 172.21.15.71:0/4133276350 wait complete. 2024-04-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.821+0000 7fe81ef74640 1 Processor -- start 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.821+0000 7fe81ef74640 1 -- start start 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81ef74640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8180713a0 0x7fe8180bf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81ef74640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8180c8130 0x7fe8180c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81ef74640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180c8ee0 0x7fe8181561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81ef74640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe818071900 con 0x7fe8180c8ee0 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81ef74640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe818071780 con 0x7fe8180713a0 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81ef74640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe818154590 con 0x7fe8180c8130 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81d511640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180c8ee0 0x7fe8181561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81d511640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180c8ee0 0x7fe8181561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55120/0 (socket says 172.21.15.71:55120) 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81d511640 1 -- 172.21.15.71:0/3927277505 learned_addr learned my addr 172.21.15.71:0/3927277505 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81cd10640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8180713a0 0x7fe8180bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81d511640 1 -- 172.21.15.71:0/3927277505 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8180c8130 msgr2=0x7fe8180c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81d511640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8180c8130 0x7fe8180c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81d511640 1 -- 172.21.15.71:0/3927277505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8180713a0 msgr2=0x7fe8180bf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81d511640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8180713a0 0x7fe8180bf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.822+0000 7fe81d511640 1 -- 172.21.15.71:0/3927277505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8180be770 con 0x7fe8180c8ee0 2024-04-16T15:41:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.823+0000 7fe81d511640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180c8ee0 0x7fe8181561d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe810007e40 tx=0x7fe81000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.823+0000 7fe80d7fa640 1 -- 172.21.15.71:0/3927277505 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810014620 con 0x7fe8180c8ee0 2024-04-16T15:41:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.823+0000 7fe80d7fa640 1 -- 172.21.15.71:0/3927277505 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe810019070 con 0x7fe8180c8ee0 2024-04-16T15:41:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.823+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8180bea50 con 0x7fe8180c8ee0 2024-04-16T15:41:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.823+0000 7fe80d7fa640 1 -- 172.21.15.71:0/3927277505 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810014a20 con 0x7fe8180c8ee0 2024-04-16T15:41:05.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.823+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8180bf010 con 0x7fe8180c8ee0 2024-04-16T15:41:05.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.824+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe818068be0 con 0x7fe8180c8ee0 2024-04-16T15:41:05.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.824+0000 7fe80d7fa640 1 -- 172.21.15.71:0/3927277505 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe810014bc0 con 0x7fe8180c8ee0 2024-04-16T15:41:05.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.825+0000 7fe80d7fa640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe80406c8f0 0x7fe80406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:05.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.825+0000 7fe80d7fa640 1 -- 172.21.15.71:0/3927277505 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fe81008c830 con 0x7fe8180c8ee0 2024-04-16T15:41:05.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.825+0000 7fe81cd10640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe80406c8f0 0x7fe80406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:05.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.825+0000 7fe81cd10640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe80406c8f0 0x7fe80406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe814036000 tx=0x7fe814008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:05.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:05.829+0000 7fe80d7fa640 1 -- 172.21.15.71:0/3927277505 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe81005a840 con 0x7fe8180c8ee0 2024-04-16T15:41:06.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.005+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe8180a8fe0 con 0x7fe8180c8ee0 2024-04-16T15:41:06.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.005+0000 7fe80d7fa640 1 -- 172.21.15.71:0/3927277505 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe81005a140 con 0x7fe8180c8ee0 2024-04-16T15:41:06.004 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe80406c8f0 msgr2=0x7fe80406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe80406c8f0 0x7fe80406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe814036000 tx=0x7fe814008030 comp rx=0 tx=0).stop 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180c8ee0 msgr2=0x7fe8181561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180c8ee0 0x7fe8181561d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe810007e40 tx=0x7fe81000e4b0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 shutdown_connections 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe80406c8f0 0x7fe80406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8180c8ee0 0x7fe8181561d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8180c8130 0x7fe8180c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 --2- 172.21.15.71:0/3927277505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8180713a0 0x7fe8180bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 >> 172.21.15.71:0/3927277505 conn(0x7fe8180c96e0 msgr2=0x7fe8180a9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:06.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 shutdown_connections 2024-04-16T15:41:06.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.007+0000 7fe81ef74640 1 -- 172.21.15.71:0/3927277505 wait complete. 2024-04-16T15:41:06.017 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T15:41:06.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.121+0000 7f170497c640 1 Processor -- start 2024-04-16T15:41:06.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.121+0000 7f170497c640 1 -- start start 2024-04-16T15:41:06.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f170497c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1700153300 0x7f1700159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f170497c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f17000caf00 0x7f17000ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f170497c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1700155c00 con 0x7f17000ca910 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f170497c640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f17001567a0 con 0x7f17000caf00 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f170497c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f170015a030 con 0x7f1700153300 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f16fdd9b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f17000caf00 0x7f17000ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f16fdd9b640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f17000caf00 0x7f17000ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55128/0 (socket says 172.21.15.71:55128) 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f16fdd9b640 1 -- 172.21.15.71:0/990083135 learned_addr learned my addr 172.21.15.71:0/990083135 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.122+0000 7f16fe59c640 1 --2- 172.21.15.71:0/990083135 >> v2:172.21.15.122:3300/0 conn(0x7f1700153300 0x7f1700159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.123+0000 7f16fdd9b640 1 -- 172.21.15.71:0/990083135 >> v2:172.21.15.122:3300/0 conn(0x7f1700153300 msgr2=0x7f1700159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.123+0000 7f16fdd9b640 1 --2- 172.21.15.71:0/990083135 >> v2:172.21.15.122:3300/0 conn(0x7f1700153300 0x7f1700159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.123+0000 7f16fdd9b640 1 -- 172.21.15.71:0/990083135 >> v1:172.21.15.122:6789/0 conn(0x7f17000ca910 legacy=0x7f17000c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.123+0000 7f16fdd9b640 1 -- 172.21.15.71:0/990083135 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1700155e70 con 0x7f17000caf00 2024-04-16T15:41:06.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.123+0000 7f16fe59c640 1 --2- 172.21.15.71:0/990083135 >> v2:172.21.15.122:3300/0 conn(0x7f1700153300 0x7f1700159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:06.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.123+0000 7f16fdd9b640 1 --2- 172.21.15.71:0/990083135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000ca3d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1700074cd0 tx=0x7f16f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c64b912dfd3efec server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.123+0000 7f16fd59a640 1 -- 172.21.15.71:0/990083135 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4035020 con 0x7f17000caf00 2024-04-16T15:41:06.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f16fd59a640 1 -- 172.21.15.71:0/990083135 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f4031870 con 0x7f17000caf00 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f16fd59a640 1 -- 172.21.15.71:0/990083135 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4031c40 con 0x7f17000caf00 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f170497c640 1 -- 172.21.15.71:0/990083135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 msgr2=0x7f17000ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f170497c640 1 --2- 172.21.15.71:0/990083135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000ca3d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1700074cd0 tx=0x7f16f402b910 comp rx=0 tx=0).stop 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f170497c640 1 -- 172.21.15.71:0/990083135 shutdown_connections 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f170497c640 1 --2- 172.21.15.71:0/990083135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000ca3d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f170497c640 1 --2- 172.21.15.71:0/990083135 >> v2:172.21.15.122:3300/0 conn(0x7f1700153300 0x7f1700159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f170497c640 1 -- 172.21.15.71:0/990083135 >> 172.21.15.71:0/990083135 conn(0x7f17000c7f40 msgr2=0x7f17000c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f170497c640 1 -- 172.21.15.71:0/990083135 shutdown_connections 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.124+0000 7f170497c640 1 -- 172.21.15.71:0/990083135 wait complete. 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.125+0000 7f170497c640 1 Processor -- start 2024-04-16T15:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f170497c640 1 -- start start 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f170497c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17000ca910 0x7f170014e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f170497c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f170497c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1700153300 0x7f1700145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f170497c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f17000c0a70 con 0x7f17000caf00 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f170497c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1700156430 con 0x7f1700153300 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f170497c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f17000c0bf0 con 0x7f17000ca910 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f16fdd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f16fdd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55134/0 (socket says 172.21.15.71:55134) 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.126+0000 7f16fdd9b640 1 -- 172.21.15.71:0/1380980319 learned_addr learned my addr 172.21.15.71:0/1380980319 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fed9d640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1700153300 0x7f1700145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fdd9b640 1 -- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17000ca910 msgr2=0x7f170014e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fe59c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17000ca910 0x7f170014e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fdd9b640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17000ca910 0x7f170014e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fdd9b640 1 -- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1700153300 msgr2=0x7f1700145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fdd9b640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1700153300 0x7f1700145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fdd9b640 1 -- 172.21.15.71:0/1380980319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17000acd10 con 0x7f17000caf00 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fed9d640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1700153300 0x7f1700145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fe59c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17000ca910 0x7f170014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16fdd9b640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000abeb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f16f4000c00 tx=0x7f16f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16eeffd640 1 -- 172.21.15.71:0/1380980319 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4035040 con 0x7f17000caf00 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16eeffd640 1 -- 172.21.15.71:0/1380980319 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f4030d00 con 0x7f17000caf00 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f16eeffd640 1 -- 172.21.15.71:0/1380980319 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40456a0 con 0x7f17000caf00 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1700155470 con 0x7f17000caf00 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.127+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1700155710 con 0x7f17000caf00 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.128+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1700068c10 con 0x7f17000caf00 2024-04-16T15:41:06.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.128+0000 7f16eeffd640 1 -- 172.21.15.71:0/1380980319 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16f4038020 con 0x7f17000caf00 2024-04-16T15:41:06.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.129+0000 7f16eeffd640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f16e006c8f0 0x7f16e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.129+0000 7f16eeffd640 1 -- 172.21.15.71:0/1380980319 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f16f40ae1e0 con 0x7f17000caf00 2024-04-16T15:41:06.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.129+0000 7f16fe59c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f16e006c8f0 0x7f16e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.129+0000 7f16fe59c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f16e006c8f0 0x7f16e006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f16f00094c0 tx=0x7f16f000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.133+0000 7f16eeffd640 1 -- 172.21.15.71:0/1380980319 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17000acd10 con 0x7f17000caf00 2024-04-16T15:41:06.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.317+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1700155910 con 0x7f17000caf00 2024-04-16T15:41:06.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.318+0000 7f16eeffd640 1 -- 172.21.15.71:0/1380980319 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f16f407c280 con 0x7f17000caf00 2024-04-16T15:41:06.317 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-04-16T15:41:06.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f16e006c8f0 msgr2=0x7f16e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f16e006c8f0 0x7f16e006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f16f00094c0 tx=0x7f16f000b040 comp rx=0 tx=0).stop 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 msgr2=0x7f17000abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000abeb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f16f4000c00 tx=0x7f16f40046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 shutdown_connections 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f16e006c8f0 0x7f16e006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1700153300 0x7f1700145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17000caf00 0x7f17000abeb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 --2- 172.21.15.71:0/1380980319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17000ca910 0x7f170014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 >> 172.21.15.71:0/1380980319 conn(0x7f17000c7f40 msgr2=0x7f17000c92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 shutdown_connections 2024-04-16T15:41:06.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.320+0000 7f170497c640 1 -- 172.21.15.71:0/1380980319 wait complete. 2024-04-16T15:41:06.329 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-16T15:41:06.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:06.330 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-16T15:41:06.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:41:06.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:06.360 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-16T15:41:06.415 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:41:06.415 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:41:06.436 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:41:06.436 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:41:06.494 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:41:06.494 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:41:06.552 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:06.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.656+0000 7f09378ba640 1 Processor -- start 2024-04-16T15:41:06.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.656+0000 7f09378ba640 1 -- start start 2024-04-16T15:41:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f09378ba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0930153220 0x7f09300c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f09378ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930073c90 0x7f093006c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f09378ba640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09300c3630 con 0x7f093006ce80 2024-04-16T15:41:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f09378ba640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f09300732e0 con 0x7f0930153220 2024-04-16T15:41:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f09378ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f09300729b0 con 0x7f0930073c90 2024-04-16T15:41:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f0934e55640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930073c90 0x7f093006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f0935656640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0930153220 0x7f09300c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f0935656640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0930153220 0x7f09300c2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:54292/0 (socket says 172.21.15.71:54292) 2024-04-16T15:41:06.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f0934e55640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930073c90 0x7f093006c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:55546/0 (socket says 172.21.15.71:55546) 2024-04-16T15:41:06.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f0935e57640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f093006ce80 0x7f0930159200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:54326/0 (socket says 172.21.15.71:54326) 2024-04-16T15:41:06.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.657+0000 7f0935656640 1 -- 172.21.15.71:0/1243576538 learned_addr learned my addr 172.21.15.71:0/1243576538 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.658+0000 7f0934e55640 1 -- 172.21.15.71:0/1243576538 >> v2:172.21.15.122:3300/0 conn(0x7f0930153220 msgr2=0x7f09300c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.658+0000 7f0934e55640 1 --2- 172.21.15.71:0/1243576538 >> v2:172.21.15.122:3300/0 conn(0x7f0930153220 0x7f09300c2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.658+0000 7f0934e55640 1 -- 172.21.15.71:0/1243576538 >> v1:172.21.15.122:6789/0 conn(0x7f093006ce80 legacy=0x7f0930159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.658+0000 7f0934e55640 1 -- 172.21.15.71:0/1243576538 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0930072c90 con 0x7f0930073c90 2024-04-16T15:41:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.658+0000 7f0935656640 1 --2- 172.21.15.71:0/1243576538 >> v2:172.21.15.122:3300/0 conn(0x7f0930153220 0x7f09300c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:06.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f0934e55640 1 --2- 172.21.15.71:0/1243576538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930073c90 0x7f093006c940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f09200097a0 tx=0x7f092002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c6a7e247562b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f0927fff640 1 -- 172.21.15.71:0/1243576538 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920035020 con 0x7f0930073c90 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f0927fff640 1 -- 172.21.15.71:0/1243576538 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0920031830 con 0x7f0930073c90 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f0927fff640 1 -- 172.21.15.71:0/1243576538 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920031c00 con 0x7f0930073c90 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f09378ba640 1 -- 172.21.15.71:0/1243576538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930073c90 msgr2=0x7f093006c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f09378ba640 1 --2- 172.21.15.71:0/1243576538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930073c90 0x7f093006c940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f09200097a0 tx=0x7f092002b8d0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f09378ba640 1 -- 172.21.15.71:0/1243576538 shutdown_connections 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f09378ba640 1 --2- 172.21.15.71:0/1243576538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930073c90 0x7f093006c940 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f09378ba640 1 --2- 172.21.15.71:0/1243576538 >> v2:172.21.15.122:3300/0 conn(0x7f0930153220 0x7f09300c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f09378ba640 1 -- 172.21.15.71:0/1243576538 >> 172.21.15.71:0/1243576538 conn(0x7f09300c4200 msgr2=0x7f09300c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f09378ba640 1 -- 172.21.15.71:0/1243576538 shutdown_connections 2024-04-16T15:41:06.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.659+0000 7f09378ba640 1 -- 172.21.15.71:0/1243576538 wait complete. 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.660+0000 7f09378ba640 1 Processor -- start 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f09378ba640 1 -- start start 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f09378ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f093006ce80 0x7f0930151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f09378ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0930073c90 0x7f0930147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f09378ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930153220 0x7f0930148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f09378ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0930153600 con 0x7f0930073c90 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f09378ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0930073770 con 0x7f093006ce80 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f09378ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f09300c3270 con 0x7f0930153220 2024-04-16T15:41:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f0934e55640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0930073c90 0x7f0930147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f0934e55640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0930073c90 0x7f0930147ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55146/0 (socket says 172.21.15.71:55146) 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f0934e55640 1 -- 172.21.15.71:0/2422538179 learned_addr learned my addr 172.21.15.71:0/2422538179 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f0935656640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f093006ce80 0x7f0930151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.661+0000 7f0935e57640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930153220 0x7f0930148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0934e55640 1 -- 172.21.15.71:0/2422538179 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930153220 msgr2=0x7f0930148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0934e55640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930153220 0x7f0930148220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0934e55640 1 -- 172.21.15.71:0/2422538179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f093006ce80 msgr2=0x7f0930151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0934e55640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f093006ce80 0x7f0930151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0934e55640 1 -- 172.21.15.71:0/2422538179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09300c7460 con 0x7f0930073c90 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0934e55640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0930073c90 0x7f0930147ce0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f09200098c0 tx=0x7f09200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0925ffb640 1 -- 172.21.15.71:0/2422538179 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920041070 con 0x7f0930073c90 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0925ffb640 1 -- 172.21.15.71:0/2422538179 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092003b920 con 0x7f0930073c90 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f0925ffb640 1 -- 172.21.15.71:0/2422538179 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920035040 con 0x7f0930073c90 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09300aca80 con 0x7f0930073c90 2024-04-16T15:41:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.662+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09300c6b50 con 0x7f0930073c90 2024-04-16T15:41:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.663+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0930068d20 con 0x7f0930073c90 2024-04-16T15:41:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.664+0000 7f0925ffb640 1 -- 172.21.15.71:0/2422538179 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f092004c060 con 0x7f0930073c90 2024-04-16T15:41:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.664+0000 7f0925ffb640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f091406c8f0 0x7f091406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.664+0000 7f0935656640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f091406c8f0 0x7f091406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.664+0000 7f0925ffb640 1 -- 172.21.15.71:0/2422538179 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f09200ad660 con 0x7f0930073c90 2024-04-16T15:41:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.665+0000 7f0935656640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f091406c8f0 0x7f091406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f092c006ca0 tx=0x7f092c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.669+0000 7f0925ffb640 1 -- 172.21.15.71:0/2422538179 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0920031800 con 0x7f0930073c90 2024-04-16T15:41:06.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.847+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f093006f1c0 con 0x7f0930073c90 2024-04-16T15:41:06.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.848+0000 7f0925ffb640 1 -- 172.21.15.71:0/2422538179 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18938 (secure 0 0 0) 0x7f092007b0b0 con 0x7f0930073c90 2024-04-16T15:41:06.847 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:06.847 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":26,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:04.856962+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:06.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f091406c8f0 msgr2=0x7f091406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f091406c8f0 0x7f091406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f092c006ca0 tx=0x7f092c009240 comp rx=0 tx=0).stop 2024-04-16T15:41:06.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0930073c90 msgr2=0x7f0930147ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0930073c90 0x7f0930147ce0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f09200098c0 tx=0x7f09200046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 shutdown_connections 2024-04-16T15:41:06.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f091406c8f0 0x7f091406edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0930153220 0x7f0930148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0930073c90 0x7f0930147ce0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 --2- 172.21.15.71:0/2422538179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f093006ce80 0x7f0930151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 >> 172.21.15.71:0/2422538179 conn(0x7f09300c4200 msgr2=0x7f093014ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:06.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 shutdown_connections 2024-04-16T15:41:06.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.850+0000 7f09378ba640 1 -- 172.21.15.71:0/2422538179 wait complete. 2024-04-16T15:41:06.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:41:06.860 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:41:06.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.969+0000 7f13edf15640 1 Processor -- start 2024-04-16T15:41:06.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.969+0000 7f13edf15640 1 -- start start 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.969+0000 7f13edf15640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f13e8153430 0x7f13e81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.969+0000 7f13edf15640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f13e8157720 0x7f13e80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.969+0000 7f13edf15640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e8156c70 con 0x7f13e8156770 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.969+0000 7f13edf15640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f13e80c67f0 con 0x7f13e8153430 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.969+0000 7f13edf15640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f13e80c5ec0 con 0x7f13e8157720 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13e77fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f13e8153430 0x7f13e81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13e7fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f13e8156770 0x7f13e81592e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33606/0 (socket says 172.21.15.71:33606) 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13e7fff640 1 -- 172.21.15.71:0/508316635 learned_addr learned my addr 172.21.15.71:0/508316635 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13dffff640 1 --2- 172.21.15.71:0/508316635 >> v2:172.21.15.122:3300/0 conn(0x7f13e8157720 0x7f13e80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13e77fe640 1 -- 172.21.15.71:0/508316635 >> v2:172.21.15.122:3300/0 conn(0x7f13e8157720 msgr2=0x7f13e80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:06.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13e77fe640 1 --2- 172.21.15.71:0/508316635 >> v2:172.21.15.122:3300/0 conn(0x7f13e8157720 0x7f13e80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13e77fe640 1 -- 172.21.15.71:0/508316635 >> v1:172.21.15.71:6789/0 conn(0x7f13e8156770 legacy=0x7f13e81592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13e77fe640 1 -- 172.21.15.71:0/508316635 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e80c61a0 con 0x7f13e8153430 2024-04-16T15:41:06.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13e6ffd640 1 -- 172.21.15.71:0/508316635 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992216158 0 0) 0x7f13e8156c70 con 0x7f13e8156770 2024-04-16T15:41:06.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.970+0000 7f13dffff640 1 --2- 172.21.15.71:0/508316635 >> v2:172.21.15.122:3300/0 conn(0x7f13e8157720 0x7f13e80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13e77fe640 1 --2- 172.21.15.71:0/508316635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e81571e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f13e8074720 tx=0x7f13d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ad52b2836d9f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13e6ffd640 1 -- 172.21.15.71:0/508316635 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80315a0 con 0x7f13e8153430 2024-04-16T15:41:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13e6ffd640 1 -- 172.21.15.71:0/508316635 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d8037070 con 0x7f13e8153430 2024-04-16T15:41:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13e6ffd640 1 -- 172.21.15.71:0/508316635 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8031960 con 0x7f13e8153430 2024-04-16T15:41:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13edf15640 1 -- 172.21.15.71:0/508316635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 msgr2=0x7f13e81571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13edf15640 1 --2- 172.21.15.71:0/508316635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e81571e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f13e8074720 tx=0x7f13d802b840 comp rx=0 tx=0).stop 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13edf15640 1 -- 172.21.15.71:0/508316635 shutdown_connections 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13edf15640 1 --2- 172.21.15.71:0/508316635 >> v2:172.21.15.122:3300/0 conn(0x7f13e8157720 0x7f13e80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13edf15640 1 --2- 172.21.15.71:0/508316635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e81571e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13edf15640 1 -- 172.21.15.71:0/508316635 >> 172.21.15.71:0/508316635 conn(0x7f13e80c7f20 msgr2=0x7f13e80c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13edf15640 1 -- 172.21.15.71:0/508316635 shutdown_connections 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.971+0000 7f13edf15640 1 -- 172.21.15.71:0/508316635 wait complete. 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.972+0000 7f13edf15640 1 Processor -- start 2024-04-16T15:41:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13edf15640 1 -- start start 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13edf15640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e80bf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13edf15640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13e8156770 0x7f13e81ede60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13edf15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13e8157720 0x7f13e80c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13edf15640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8153990 con 0x7f13e8153430 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13edf15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8153810 con 0x7f13e8157720 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13edf15640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f13e80cb120 con 0x7f13e8156770 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13e77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e80bf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13e77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e80bf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55164/0 (socket says 172.21.15.71:55164) 2024-04-16T15:41:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13e77fe640 1 -- 172.21.15.71:0/2074087400 learned_addr learned my addr 172.21.15.71:0/2074087400 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13e7fff640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13e8157720 0x7f13e80c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.973+0000 7f13dffff640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13e8156770 0x7f13e81ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e77fe640 1 -- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13e8156770 msgr2=0x7f13e81ede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e77fe640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13e8156770 0x7f13e81ede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e77fe640 1 -- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13e8157720 msgr2=0x7f13e80c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e77fe640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13e8157720 0x7f13e80c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e77fe640 1 -- 172.21.15.71:0/2074087400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e81ee3a0 con 0x7f13e8153430 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e7fff640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13e8157720 0x7f13e80c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e77fe640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e80bf190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009890 tx=0x7f13d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e4ff9640 1 -- 172.21.15.71:0/2074087400 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8032840 con 0x7f13e8153430 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e4ff9640 1 -- 172.21.15.71:0/2074087400 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d80329e0 con 0x7f13e8153430 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e80c3440 con 0x7f13e8153430 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13e4ff9640 1 -- 172.21.15.71:0/2074087400 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8032dd0 con 0x7f13e8153430 2024-04-16T15:41:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.974+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e80c39c0 con 0x7f13e8153430 2024-04-16T15:41:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.975+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e8073f50 con 0x7f13e8153430 2024-04-16T15:41:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.975+0000 7f13e4ff9640 1 -- 172.21.15.71:0/2074087400 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13d804c020 con 0x7f13e8153430 2024-04-16T15:41:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.976+0000 7f13e4ff9640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f13c806c920 0x7f13c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.976+0000 7f13e4ff9640 1 -- 172.21.15.71:0/2074087400 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f13d802f020 con 0x7f13e8153430 2024-04-16T15:41:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.976+0000 7f13dffff640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f13c806c920 0x7f13c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.976+0000 7f13dffff640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f13c806c920 0x7f13c806ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13e8155f50 tx=0x7f13d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:06.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:06.980+0000 7f13e4ff9640 1 -- 172.21.15.71:0/2074087400 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13d8040070 con 0x7f13e8153430 2024-04-16T15:41:07.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.155+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13e8068be0 con 0x7f13e8153430 2024-04-16T15:41:07.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.155+0000 7f13e4ff9640 1 -- 172.21.15.71:0/2074087400 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f13d807bd70 con 0x7f13e8153430 2024-04-16T15:41:07.154 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:41:07.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f13c806c920 msgr2=0x7f13c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f13c806c920 0x7f13c806ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13e8155f50 tx=0x7f13d0008040 comp rx=0 tx=0).stop 2024-04-16T15:41:07.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 msgr2=0x7f13e80bf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e80bf190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009890 tx=0x7f13d80046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 shutdown_connections 2024-04-16T15:41:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f13c806c920 0x7f13c806ede0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13e8157720 0x7f13e80c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13e8156770 0x7f13e81ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 --2- 172.21.15.71:0/2074087400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8153430 0x7f13e80bf190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 >> 172.21.15.71:0/2074087400 conn(0x7f13e80c7f20 msgr2=0x7f13e8151cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 shutdown_connections 2024-04-16T15:41:07.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.157+0000 7f13edf15640 1 -- 172.21.15.71:0/2074087400 wait complete. 2024-04-16T15:41:07.167 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:41:07.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.274+0000 7f6869700640 1 Processor -- start 2024-04-16T15:41:07.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.274+0000 7f6869700640 1 -- start start 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.274+0000 7f6869700640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f68640b0f40 0x7f68640c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.274+0000 7f6869700640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6864156cc0 con 0x7f6864156820 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.274+0000 7f6869700640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68640c8170 con 0x7f68640c9ad0 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.274+0000 7f6869700640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f68640c8450 con 0x7f68640b0f40 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f68637fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f6864156820 0x7f686406d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:33618/0 (socket says 172.21.15.71:33618) 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f68637fe640 1 -- 172.21.15.71:0/3742850372 learned_addr learned my addr 172.21.15.71:0/3742850372 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f6862ffd640 1 --2- 172.21.15.71:0/3742850372 >> v2:172.21.15.122:3300/0 conn(0x7f68640b0f40 0x7f68640c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f6861ffb640 1 -- 172.21.15.71:0/3742850372 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569785529 0 0) 0x7f6864156cc0 con 0x7f6864156820 2024-04-16T15:41:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f6861ffb640 1 -- 172.21.15.71:0/3742850372 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f684c003520 con 0x7f6864156820 2024-04-16T15:41:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f6862ffd640 1 -- 172.21.15.71:0/3742850372 >> v1:172.21.15.122:6789/0 conn(0x7f68640c9ad0 legacy=0x7f686406cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f6862ffd640 1 -- 172.21.15.71:0/3742850372 >> v1:172.21.15.71:6789/0 conn(0x7f6864156820 legacy=0x7f686406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f6861ffb640 1 -- 172.21.15.71:0/3742850372 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3900642774 0 0) 0x7f684c003520 con 0x7f6864156820 2024-04-16T15:41:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.275+0000 7f6862ffd640 1 -- 172.21.15.71:0/3742850372 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68640c8730 con 0x7f68640b0f40 2024-04-16T15:41:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.276+0000 7f6862ffd640 1 --2- 172.21.15.71:0/3742850372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640b0f40 0x7f68640c6040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f68540097d0 tx=0x7f685402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff0c95ce2a4fd0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.276+0000 7f6861ffb640 1 -- 172.21.15.71:0/3742850372 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854037070 con 0x7f68640b0f40 2024-04-16T15:41:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.276+0000 7f6861ffb640 1 -- 172.21.15.71:0/3742850372 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6854030510 con 0x7f68640b0f40 2024-04-16T15:41:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.276+0000 7f6861ffb640 1 -- 172.21.15.71:0/3742850372 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c0019f0 con 0x7f68640b0f40 2024-04-16T15:41:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.277+0000 7f6869700640 1 -- 172.21.15.71:0/3742850372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640b0f40 msgr2=0x7f68640c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.277+0000 7f6869700640 1 --2- 172.21.15.71:0/3742850372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640b0f40 0x7f68640c6040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f68540097d0 tx=0x7f685402b710 comp rx=0 tx=0).stop 2024-04-16T15:41:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.277+0000 7f6869700640 1 -- 172.21.15.71:0/3742850372 shutdown_connections 2024-04-16T15:41:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.277+0000 7f6869700640 1 --2- 172.21.15.71:0/3742850372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640b0f40 0x7f68640c6040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.277+0000 7f6869700640 1 -- 172.21.15.71:0/3742850372 >> 172.21.15.71:0/3742850372 conn(0x7f68640ad3f0 msgr2=0x7f68640ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:07.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.277+0000 7f6869700640 1 -- 172.21.15.71:0/3742850372 shutdown_connections 2024-04-16T15:41:07.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.277+0000 7f6869700640 1 -- 172.21.15.71:0/3742850372 wait complete. 2024-04-16T15:41:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.278+0000 7f6869700640 1 Processor -- start 2024-04-16T15:41:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.278+0000 7f6869700640 1 -- start start 2024-04-16T15:41:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6869700640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68640b0f40 0x7f68641eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6869700640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640c9ad0 0x7f686414de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6869700640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6864156820 0x7f6864159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6869700640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68640c6580 con 0x7f68640b0f40 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6869700640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68640c97d0 con 0x7f68640c9ad0 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6869700640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f68640c6700 con 0x7f6864156820 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68640b0f40 0x7f68641eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68640b0f40 0x7f68641eec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55172/0 (socket says 172.21.15.71:55172) 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 -- 172.21.15.71:0/739471048 learned_addr learned my addr 172.21.15.71:0/739471048 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f68627fc640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640c9ad0 0x7f686414de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f68637fe640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6864156820 0x7f6864159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 -- 172.21.15.71:0/739471048 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6864156820 msgr2=0x7f6864159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6864156820 0x7f6864159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 -- 172.21.15.71:0/739471048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640c9ad0 msgr2=0x7f686414de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640c9ad0 0x7f686414de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 -- 172.21.15.71:0/739471048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68640c3be0 con 0x7f68640b0f40 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.279+0000 7f6862ffd640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68640b0f40 0x7f68641eec70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f685402b750 tx=0x7f68540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.280+0000 7f684b7fe640 1 -- 172.21.15.71:0/739471048 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854042020 con 0x7f68640b0f40 2024-04-16T15:41:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.280+0000 7f684b7fe640 1 -- 172.21.15.71:0/739471048 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68540044c0 con 0x7f68640b0f40 2024-04-16T15:41:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.280+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68640c3ec0 con 0x7f68640b0f40 2024-04-16T15:41:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.280+0000 7f684b7fe640 1 -- 172.21.15.71:0/739471048 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854037040 con 0x7f68640b0f40 2024-04-16T15:41:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.280+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686414e3d0 con 0x7f68640b0f40 2024-04-16T15:41:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.281+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6864068be0 con 0x7f68640b0f40 2024-04-16T15:41:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.281+0000 7f684b7fe640 1 -- 172.21.15.71:0/739471048 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6854045020 con 0x7f68640b0f40 2024-04-16T15:41:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.281+0000 7f684b7fe640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f685006c8f0 0x7f685006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.281+0000 7f684b7fe640 1 -- 172.21.15.71:0/739471048 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f68540adb10 con 0x7f68640b0f40 2024-04-16T15:41:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.281+0000 7f68627fc640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f685006c8f0 0x7f685006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.282+0000 7f68627fc640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f685006c8f0 0x7f685006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f686415a4e0 tx=0x7f6858009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.286+0000 7f684b7fe640 1 -- 172.21.15.71:0/739471048 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f685404c030 con 0x7f68640b0f40 2024-04-16T15:41:07.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.465+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f686406e4c0 con 0x7f68640b0f40 2024-04-16T15:41:07.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.465+0000 7f684b7fe640 1 -- 172.21.15.71:0/739471048 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f685407b560 con 0x7f68640b0f40 2024-04-16T15:41:07.464 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:07.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f685006c8f0 msgr2=0x7f685006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f685006c8f0 0x7f685006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f686415a4e0 tx=0x7f6858009040 comp rx=0 tx=0).stop 2024-04-16T15:41:07.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68640b0f40 msgr2=0x7f68641eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68640b0f40 0x7f68641eec70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f685402b750 tx=0x7f68540046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 shutdown_connections 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f685006c8f0 0x7f685006edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6864156820 0x7f6864159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68640c9ad0 0x7f686414de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 --2- 172.21.15.71:0/739471048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68640b0f40 0x7f68641eec70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 >> 172.21.15.71:0/739471048 conn(0x7f68640ad3f0 msgr2=0x7f68640ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 shutdown_connections 2024-04-16T15:41:07.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.467+0000 7f6869700640 1 -- 172.21.15.71:0/739471048 wait complete. 2024-04-16T15:41:07.477 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:41:07.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.584+0000 7f31584d9640 1 Processor -- start 2024-04-16T15:41:07.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.584+0000 7f31584d9640 1 -- start start 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f31584d9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f315006e0a0 0x7f3150154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f31584d9640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31500ace00 con 0x7f31500ac5c0 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f31584d9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31500c3e20 con 0x7f31500ac9a0 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f31584d9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f31501543a0 con 0x7f315006e0a0 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3156275640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f31500ac5c0 0x7f31500c3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48080/0 (socket says 172.21.15.71:48080) 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3156275640 1 -- 172.21.15.71:0/545850576 learned_addr learned my addr 172.21.15.71:0/545850576 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155a74640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f315006e0a0 0x7f3150154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969890573 0 0) 0x7f31500ace00 con 0x7f31500ac5c0 2024-04-16T15:41:07.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f313c003520 con 0x7f31500ac5c0 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1832817281 0 0) 0x7f313c003520 con 0x7f31500ac5c0 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f315006e0a0 msgr2=0x7f3150154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f315006e0a0 0x7f3150154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 >> v1:172.21.15.122:6789/0 conn(0x7f31500ac9a0 legacy=0x7f315015e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31500a9e90 con 0x7f31500ac5c0 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f314c001670 con 0x7f31500ac5c0 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 >> v1:172.21.15.71:6789/0 conn(0x7f31500ac5c0 legacy=0x7f31500c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f313c0039e0 0x7f313c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f315006e0a0 0x7f313c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c004d40 0x7f313c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f31501543a0 con 0x7f313c004d40 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f313c00b9c0 con 0x7f313c0039e0 2024-04-16T15:41:07.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.585+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f313c00bbc0 con 0x7f315006e0a0 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.586+0000 7f3156a76640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c004d40 0x7f313c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.586+0000 7f3155a74640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f315006e0a0 0x7f313c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.586+0000 7f3156275640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f313c0039e0 0x7f313c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.586+0000 7f3156a76640 1 -- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f315006e0a0 msgr2=0x7f313c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.586+0000 7f3156a76640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f315006e0a0 0x7f313c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.586+0000 7f3156a76640 1 -- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f313c0039e0 msgr2=0x7f313c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.586+0000 7f3156a76640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f313c0039e0 0x7f313c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.586+0000 7f3156a76640 1 -- 172.21.15.71:0/545850576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c003520 con 0x7f313c004d40 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f3156a76640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c004d40 0x7f313c0051c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3148002010 tx=0x7f313c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22accd697310c237 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148039070 con 0x7f313c004d40 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314802c750 con 0x7f313c004d40 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f3155273640 1 -- 172.21.15.71:0/545850576 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148001be0 con 0x7f313c004d40 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f31584d9640 1 -- 172.21.15.71:0/545850576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c004d40 msgr2=0x7f313c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f31584d9640 1 --2- 172.21.15.71:0/545850576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c004d40 0x7f313c0051c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3148002010 tx=0x7f313c005830 comp rx=0 tx=0).stop 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f3156a76640 1 -- 172.21.15.71:0/545850576 reap_dead start 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f31584d9640 1 -- 172.21.15.71:0/545850576 shutdown_connections 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f31584d9640 1 -- 172.21.15.71:0/545850576 >> 172.21.15.71:0/545850576 conn(0x7f31500c8470 msgr2=0x7f31500c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f31584d9640 1 -- 172.21.15.71:0/545850576 shutdown_connections 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.587+0000 7f31584d9640 1 -- 172.21.15.71:0/545850576 wait complete. 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f31584d9640 1 Processor -- start 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f31584d9640 1 -- start start 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f31584d9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f313c004d40 0x7f31501ed410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f31584d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315014e1b0 0x7f315006d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f31584d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31500bf6a0 0x7f315014ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f31584d9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3150154030 con 0x7f31500bf6a0 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f31584d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31500aa450 con 0x7f315014e1b0 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f31584d9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3150153d00 con 0x7f313c004d40 2024-04-16T15:41:07.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f3156a76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31500bf6a0 0x7f315014ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f3156a76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31500bf6a0 0x7f315014ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57408/0 (socket says 172.21.15.71:57408) 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.589+0000 7f3156a76640 1 -- 172.21.15.71:0/2275446225 learned_addr learned my addr 172.21.15.71:0/2275446225 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3155a74640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315014e1b0 0x7f315006d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3156275640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f313c004d40 0x7f31501ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3156a76640 1 -- 172.21.15.71:0/2275446225 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f313c004d40 msgr2=0x7f31501ed410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3156a76640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f313c004d40 0x7f31501ed410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3156a76640 1 -- 172.21.15.71:0/2275446225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315014e1b0 msgr2=0x7f315006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3156a76640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315014e1b0 0x7f315006d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3156a76640 1 -- 172.21.15.71:0/2275446225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315014ded0 con 0x7f31500bf6a0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3156a76640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31500bf6a0 0x7f315014ccd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3148002130 tx=0x7f3148001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3146ffd640 1 -- 172.21.15.71:0/2275446225 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148045070 con 0x7f31500bf6a0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3146ffd640 1 -- 172.21.15.71:0/2275446225 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314802de50 con 0x7f31500bf6a0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31501ecc10 con 0x7f31500bf6a0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f3146ffd640 1 -- 172.21.15.71:0/2275446225 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3148039040 con 0x7f31500bf6a0 2024-04-16T15:41:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.590+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3150156390 con 0x7f31500bf6a0 2024-04-16T15:41:07.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.591+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31501eca00 con 0x7f31500bf6a0 2024-04-16T15:41:07.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.591+0000 7f3146ffd640 1 -- 172.21.15.71:0/2275446225 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3148048070 con 0x7f31500bf6a0 2024-04-16T15:41:07.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.592+0000 7f3146ffd640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f313406c8f0 0x7f313406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.592+0000 7f3146ffd640 1 -- 172.21.15.71:0/2275446225 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f31480accd0 con 0x7f31500bf6a0 2024-04-16T15:41:07.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.592+0000 7f3156275640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f313406c8f0 0x7f313406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.592+0000 7f3156275640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f313406c8f0 0x7f313406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f314c000c00 tx=0x7f314c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.596+0000 7f3146ffd640 1 -- 172.21.15.71:0/2275446225 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f314807a690 con 0x7f31500bf6a0 2024-04-16T15:41:07.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.772+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3150068be0 con 0x7f31500bf6a0 2024-04-16T15:41:07.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.772+0000 7f3146ffd640 1 -- 172.21.15.71:0/2275446225 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f314807e990 con 0x7f31500bf6a0 2024-04-16T15:41:07.771 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:41:07.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f313406c8f0 msgr2=0x7f313406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f313406c8f0 0x7f313406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f314c000c00 tx=0x7f314c009040 comp rx=0 tx=0).stop 2024-04-16T15:41:07.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31500bf6a0 msgr2=0x7f315014ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31500bf6a0 0x7f315014ccd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3148002130 tx=0x7f3148001630 comp rx=0 tx=0).stop 2024-04-16T15:41:07.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 shutdown_connections 2024-04-16T15:41:07.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f313406c8f0 0x7f313406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31500bf6a0 0x7f315014ccd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315014e1b0 0x7f315006d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 --2- 172.21.15.71:0/2275446225 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f313c004d40 0x7f31501ed410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 >> 172.21.15.71:0/2275446225 conn(0x7f31500c8470 msgr2=0x7f3150152350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:07.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 shutdown_connections 2024-04-16T15:41:07.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.774+0000 7f31584d9640 1 -- 172.21.15.71:0/2275446225 wait complete. 2024-04-16T15:41:07.784 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:41:07.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.891+0000 7f52c424e640 1 Processor -- start 2024-04-16T15:41:07.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.892+0000 7f52c424e640 1 -- start start 2024-04-16T15:41:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.892+0000 7f52c424e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc073730 0x7f52bc073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.892+0000 7f52c424e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f52bc153970 0x7f52bc153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.892+0000 7f52c424e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52bc06fc10 con 0x7f52bc14dba0 2024-04-16T15:41:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.892+0000 7f52c424e640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f52bc06f9c0 con 0x7f52bc153970 2024-04-16T15:41:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.892+0000 7f52c424e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc070540 con 0x7f52bc073730 2024-04-16T15:41:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c1fea640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f52bc14dba0 0x7f52bc072a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48092/0 (socket says 172.21.15.71:48092) 2024-04-16T15:41:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c1fea640 1 -- 172.21.15.71:0/998871637 learned_addr learned my addr 172.21.15.71:0/998871637 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c27eb640 1 --2- 172.21.15.71:0/998871637 >> v2:172.21.15.71:3300/0 conn(0x7f52bc153970 0x7f52bc153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c17e9640 1 --2- 172.21.15.71:0/998871637 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc073730 0x7f52bc073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c0fe8640 1 -- 172.21.15.71:0/998871637 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123083253 0 0) 0x7f52bc06fc10 con 0x7f52bc14dba0 2024-04-16T15:41:07.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c0fe8640 1 -- 172.21.15.71:0/998871637 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52a8003520 con 0x7f52bc14dba0 2024-04-16T15:41:07.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c27eb640 1 -- 172.21.15.71:0/998871637 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc073730 msgr2=0x7f52bc073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:07.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c27eb640 1 --2- 172.21.15.71:0/998871637 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc073730 0x7f52bc073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c27eb640 1 -- 172.21.15.71:0/998871637 >> v1:172.21.15.71:6789/0 conn(0x7f52bc14dba0 legacy=0x7f52bc072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.893+0000 7f52c27eb640 1 -- 172.21.15.71:0/998871637 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc06fef0 con 0x7f52bc153970 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c27eb640 1 --2- 172.21.15.71:0/998871637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc153970 0x7f52bc153dc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009d90 tx=0x7f52b402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1737f278574019cd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c0fe8640 1 -- 172.21.15.71:0/998871637 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4039070 con 0x7f52bc153970 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c0fe8640 1 -- 172.21.15.71:0/998871637 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b402cbc0 con 0x7f52bc153970 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c0fe8640 1 -- 172.21.15.71:0/998871637 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4034440 con 0x7f52bc153970 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c424e640 1 -- 172.21.15.71:0/998871637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc153970 msgr2=0x7f52bc153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c424e640 1 --2- 172.21.15.71:0/998871637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc153970 0x7f52bc153dc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009d90 tx=0x7f52b402be10 comp rx=0 tx=0).stop 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c424e640 1 -- 172.21.15.71:0/998871637 shutdown_connections 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c424e640 1 --2- 172.21.15.71:0/998871637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc153970 0x7f52bc153dc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c424e640 1 --2- 172.21.15.71:0/998871637 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc073730 0x7f52bc073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.894+0000 7f52c424e640 1 -- 172.21.15.71:0/998871637 >> 172.21.15.71:0/998871637 conn(0x7f52bc06efa0 msgr2=0x7f52bc0c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.895+0000 7f52c424e640 1 -- 172.21.15.71:0/998871637 shutdown_connections 2024-04-16T15:41:07.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.895+0000 7f52c424e640 1 -- 172.21.15.71:0/998871637 wait complete. 2024-04-16T15:41:07.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c424e640 1 Processor -- start 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c424e640 1 -- start start 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c424e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc073730 0x7f52bc1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c424e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc14dba0 0x7f52bc150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c424e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc153970 0x7f52bc0c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c424e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc073170 con 0x7f52bc073730 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c424e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc0c5860 con 0x7f52bc153970 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c424e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc154300 con 0x7f52bc14dba0 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c1fea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc073730 0x7f52bc1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c1fea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc073730 0x7f52bc1506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57420/0 (socket says 172.21.15.71:57420) 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.896+0000 7f52c1fea640 1 -- 172.21.15.71:0/754325006 learned_addr learned my addr 172.21.15.71:0/754325006 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52c1fea640 1 -- 172.21.15.71:0/754325006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc14dba0 msgr2=0x7f52bc150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52c1fea640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc14dba0 0x7f52bc150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52c1fea640 1 -- 172.21.15.71:0/754325006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc153970 msgr2=0x7f52bc0c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:07.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52c1fea640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc153970 0x7f52bc0c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:07.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52c1fea640 1 -- 172.21.15.71:0/754325006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc1e9f10 con 0x7f52bc073730 2024-04-16T15:41:07.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52c1fea640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc073730 0x7f52bc1506a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f52b800c430 tx=0x7f52b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52b27fc640 1 -- 172.21.15.71:0/754325006 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8017070 con 0x7f52bc073730 2024-04-16T15:41:07.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52b27fc640 1 -- 172.21.15.71:0/754325006 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b8005ce0 con 0x7f52bc073730 2024-04-16T15:41:07.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc14e0b0 con 0x7f52bc073730 2024-04-16T15:41:07.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52b27fc640 1 -- 172.21.15.71:0/754325006 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b80072f0 con 0x7f52bc073730 2024-04-16T15:41:07.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.897+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc146de0 con 0x7f52bc073730 2024-04-16T15:41:07.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.898+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52bc068be0 con 0x7f52bc073730 2024-04-16T15:41:07.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.898+0000 7f52b27fc640 1 -- 172.21.15.71:0/754325006 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52b801a070 con 0x7f52bc073730 2024-04-16T15:41:07.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.899+0000 7f52b27fc640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52a006c8f0 0x7f52a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:07.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.899+0000 7f52b27fc640 1 -- 172.21.15.71:0/754325006 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f52b808b3c0 con 0x7f52bc073730 2024-04-16T15:41:07.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.899+0000 7f52c17e9640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52a006c8f0 0x7f52a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:07.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.899+0000 7f52c17e9640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52a006c8f0 0x7f52a006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52bc152ff0 tx=0x7f52ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:07.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:07.903+0000 7f52b27fc640 1 -- 172.21.15.71:0/754325006 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52b8059460 con 0x7f52bc073730 2024-04-16T15:41:08.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.079+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f52bc0c8f90 con 0x7f52bc073730 2024-04-16T15:41:08.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.080+0000 7f52b27fc640 1 -- 172.21.15.71:0/754325006 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f52b8058d60 con 0x7f52bc073730 2024-04-16T15:41:08.079 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52a006c8f0 msgr2=0x7f52a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52a006c8f0 0x7f52a006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52bc152ff0 tx=0x7f52ac008040 comp rx=0 tx=0).stop 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc073730 msgr2=0x7f52bc1506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc073730 0x7f52bc1506a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f52b800c430 tx=0x7f52b800c8f0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 shutdown_connections 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52a006c8f0 0x7f52a006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc153970 0x7f52bc0c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52bc14dba0 0x7f52bc150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 --2- 172.21.15.71:0/754325006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52bc073730 0x7f52bc1506a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 >> 172.21.15.71:0/754325006 conn(0x7f52bc06efa0 msgr2=0x7f52bc151610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:08.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 shutdown_connections 2024-04-16T15:41:08.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.082+0000 7f52c424e640 1 -- 172.21.15.71:0/754325006 wait complete. 2024-04-16T15:41:08.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T15:41:08.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.196+0000 7fc114bf4640 1 Processor -- start 2024-04-16T15:41:08.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.196+0000 7fc114bf4640 1 -- start start 2024-04-16T15:41:08.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc114bf4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110156b70 0x7fc11006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc114bf4640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11014cd80 con 0x7fc11014c9a0 2024-04-16T15:41:08.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc114bf4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1100c2f30 con 0x7fc11006e670 2024-04-16T15:41:08.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc114bf4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc110155fd0 con 0x7fc110156b70 2024-04-16T15:41:08.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10e59c640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fc11014c9a0 0x7fc110153330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48100/0 (socket says 172.21.15.71:48100) 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10e59c640 1 -- 172.21.15.71:0/251552928 learned_addr learned my addr 172.21.15.71:0/251552928 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10ed9d640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110156b70 0x7fc11006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2281240112 0 0) 0x7fc11014cd80 con 0x7fc11014c9a0 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0fc003520 con 0x7fc11014c9a0 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1147387062 0 0) 0x7fc0fc003520 con 0x7fc11014c9a0 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110156b70 msgr2=0x7fc11006eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110156b70 0x7fc11006eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 >> v1:172.21.15.122:6789/0 conn(0x7fc11006e670 legacy=0x7fc110156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1100c31d0 con 0x7fc11014c9a0 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fc100001670 con 0x7fc11014c9a0 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 >> v1:172.21.15.71:6789/0 conn(0x7fc11014c9a0 legacy=0x7fc110153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc0039e0 0x7fc0fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0fc004800 0x7fc0fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110156b70 0x7fc0fc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc110155fd0 con 0x7fc0fc004800 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc00dd30 con 0x7fc0fc0039e0 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc0fc00df30 con 0x7fc110156b70 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc105d9b640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0fc004800 0x7fc0fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10e59c640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc0039e0 0x7fc0fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.197+0000 7fc10ed9d640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110156b70 0x7fc0fc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.198+0000 7fc105d9b640 1 -- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110156b70 msgr2=0x7fc0fc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.198+0000 7fc105d9b640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110156b70 0x7fc0fc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.198+0000 7fc105d9b640 1 -- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc0039e0 msgr2=0x7fc0fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.198+0000 7fc105d9b640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc0039e0 0x7fc0fc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.198+0000 7fc105d9b640 1 -- 172.21.15.71:0/251552928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc003520 con 0x7fc0fc004800 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.198+0000 7fc105d9b640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0fc004800 0x7fc0fc004c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8001750 tx=0x7fc0fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266c702618bc87b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8001370 con 0x7fc0fc004800 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f80052d0 con 0x7fc0fc004800 2024-04-16T15:41:08.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc10dd9b640 1 -- 172.21.15.71:0/251552928 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8003830 con 0x7fc0fc004800 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc114bf4640 1 -- 172.21.15.71:0/251552928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0fc004800 msgr2=0x7fc0fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc114bf4640 1 --2- 172.21.15.71:0/251552928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0fc004800 0x7fc0fc004c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8001750 tx=0x7fc0fc005830 comp rx=0 tx=0).stop 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc10ed9d640 1 -- 172.21.15.71:0/251552928 reap_dead start 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc114bf4640 1 -- 172.21.15.71:0/251552928 shutdown_connections 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc114bf4640 1 -- 172.21.15.71:0/251552928 >> 172.21.15.71:0/251552928 conn(0x7fc11006e220 msgr2=0x7fc1100c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc114bf4640 1 -- 172.21.15.71:0/251552928 shutdown_connections 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.199+0000 7fc114bf4640 1 -- 172.21.15.71:0/251552928 wait complete. 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc114bf4640 1 Processor -- start 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc114bf4640 1 -- start start 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc114bf4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc004800 0x7fc11014bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc114bf4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110155300 0x7fc110153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc114bf4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc110155760 0x7fc1101542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc114bf4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc1100c3a70 con 0x7fc110155760 2024-04-16T15:41:08.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc114bf4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1100c3790 con 0x7fc0fc004800 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc114bf4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc1100aa3c0 con 0x7fc110155300 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc10ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc110155760 0x7fc1101542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc10ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc110155760 0x7fc1101542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57434/0 (socket says 172.21.15.71:57434) 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.201+0000 7fc10ed9d640 1 -- 172.21.15.71:0/880674665 learned_addr learned my addr 172.21.15.71:0/880674665 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc105d9b640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110155300 0x7fc110153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc10e59c640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc004800 0x7fc11014bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc10ed9d640 1 -- 172.21.15.71:0/880674665 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110155300 msgr2=0x7fc110153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc10ed9d640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110155300 0x7fc110153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc10ed9d640 1 -- 172.21.15.71:0/880674665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc004800 msgr2=0x7fc11014bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc10ed9d640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc004800 0x7fc11014bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc10ed9d640 1 -- 172.21.15.71:0/880674665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11014a4f0 con 0x7fc110155760 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc10ed9d640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc110155760 0x7fc1101542b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc1101528b0 tx=0x7fc10800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc1077fe640 1 -- 172.21.15.71:0/880674665 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108015430 con 0x7fc110155760 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc1077fe640 1 -- 172.21.15.71:0/880674665 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10801d070 con 0x7fc110155760 2024-04-16T15:41:08.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11014a7d0 con 0x7fc110155760 2024-04-16T15:41:08.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc1077fe640 1 -- 172.21.15.71:0/880674665 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108015830 con 0x7fc110155760 2024-04-16T15:41:08.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.202+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11014ad90 con 0x7fc110155760 2024-04-16T15:41:08.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.203+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc110068c10 con 0x7fc110155760 2024-04-16T15:41:08.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.204+0000 7fc1077fe640 1 -- 172.21.15.71:0/880674665 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1080159d0 con 0x7fc110155760 2024-04-16T15:41:08.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.204+0000 7fc1077fe640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc0f006c920 0x7fc0f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.204+0000 7fc1077fe640 1 -- 172.21.15.71:0/880674665 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fc10808c8f0 con 0x7fc110155760 2024-04-16T15:41:08.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.204+0000 7fc10e59c640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc0f006c920 0x7fc0f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.205+0000 7fc10e59c640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc0f006c920 0x7fc0f006ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1000011e0 tx=0x7fc100000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:08.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.209+0000 7fc1077fe640 1 -- 172.21.15.71:0/880674665 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc108010040 con 0x7fc110155760 2024-04-16T15:41:08.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.395+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc11006fe60 con 0x7fc110155760 2024-04-16T15:41:08.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.395+0000 7fc1077fe640 1 -- 172.21.15.71:0/880674665 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fc10808ae20 con 0x7fc110155760 2024-04-16T15:41:08.394 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2024-04-16T15:41:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc0f006c920 msgr2=0x7fc0f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc0f006c920 0x7fc0f006ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1000011e0 tx=0x7fc100000ed0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc110155760 msgr2=0x7fc1101542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc110155760 0x7fc1101542b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc1101528b0 tx=0x7fc10800f580 comp rx=0 tx=0).stop 2024-04-16T15:41:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 shutdown_connections 2024-04-16T15:41:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fc0f006c920 0x7fc0f006ede0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc110155760 0x7fc1101542b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc110155300 0x7fc110153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 --2- 172.21.15.71:0/880674665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0fc004800 0x7fc11014bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 >> 172.21.15.71:0/880674665 conn(0x7fc11006e220 msgr2=0x7fc110062440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 shutdown_connections 2024-04-16T15:41:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.397+0000 7fc114bf4640 1 -- 172.21.15.71:0/880674665 wait complete. 2024-04-16T15:41:08.407 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-16T15:41:08.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:08.407 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-16T15:41:08.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:41:08.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:08.444 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-16T15:41:08.603 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-04-16T15:41:08.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:41:08.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.708+0000 7f6c1ecba640 1 Processor -- start 2024-04-16T15:41:08.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.708+0000 7f6c1ecba640 1 -- start start 2024-04-16T15:41:08.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1ecba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6c18153450 0x7f6c180c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1ecba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 0x7f6c18155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1ecba640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c1806e820 con 0x7f6c180c94c0 2024-04-16T15:41:08.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1ecba640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6c18159fb0 con 0x7f6c18153450 2024-04-16T15:41:08.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1ecba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6c180c8150 con 0x7f6c180c99d0 2024-04-16T15:41:08.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1d257640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f6c180c94c0 0x7f6c181562f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48104/0 (socket says 172.21.15.71:48104) 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1d257640 1 -- 172.21.15.71:0/3451307064 learned_addr learned my addr 172.21.15.71:0/3451307064 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0ffff640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 0x7f6c18155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1289760157 0 0) 0x7f6c1806e820 con 0x7f6c180c94c0 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c00003520 con 0x7f6c180c94c0 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1ca56640 1 --2- 172.21.15.71:0/3451307064 >> v2:172.21.15.122:3300/0 conn(0x7f6c18153450 0x7f6c180c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4207495668 0 0) 0x7f6c00003520 con 0x7f6c180c94c0 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 msgr2=0x7f6c18155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 0x7f6c18155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 >> v2:172.21.15.122:3300/0 conn(0x7f6c18153450 msgr2=0x7f6c180c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 --2- 172.21.15.71:0/3451307064 >> v2:172.21.15.122:3300/0 conn(0x7f6c18153450 0x7f6c180c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c180c8430 con 0x7f6c180c94c0 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f6c10001670 con 0x7f6c180c94c0 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0ffff640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 0x7f6c18155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c1ca56640 1 --2- 172.21.15.71:0/3451307064 >> v2:172.21.15.122:3300/0 conn(0x7f6c18153450 0x7f6c180c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:08.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 >> v1:172.21.15.71:6789/0 conn(0x7f6c180c94c0 legacy=0x7f6c181562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c000039b0 0x7f6c00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.709+0000 7f6c0f7fe640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c00004800 0x7f6c00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0f7fe640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 0x7f6c00005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18159fb0 con 0x7f6c00004800 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c180c8150 con 0x7f6c000039b0 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0000bb80 con 0x7f6c180c99d0 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0ffff640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c00004800 0x7f6c00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c1ca56640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c000039b0 0x7f6c00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c1d257640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 0x7f6c00005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0ffff640 1 -- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 msgr2=0x7f6c00005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0ffff640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c180c99d0 0x7f6c00005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0ffff640 1 -- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c000039b0 msgr2=0x7f6c00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0ffff640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c000039b0 0x7f6c00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.710+0000 7f6c0ffff640 1 -- 172.21.15.71:0/3451307064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c00003520 con 0x7f6c00004800 2024-04-16T15:41:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c0ffff640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c00004800 0x7f6c00004c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c080098a0 tx=0x7f6c00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcf5b7439bf390 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0800f040 con 0x7f6c00004800 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c08004230 con 0x7f6c00004800 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c0f7fe640 1 -- 172.21.15.71:0/3451307064 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08003610 con 0x7f6c00004800 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c1ecba640 1 -- 172.21.15.71:0/3451307064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c00004800 msgr2=0x7f6c00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c1ecba640 1 --2- 172.21.15.71:0/3451307064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c00004800 0x7f6c00004c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c080098a0 tx=0x7f6c00005830 comp rx=0 tx=0).stop 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c1d257640 1 -- 172.21.15.71:0/3451307064 reap_dead start 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c1ecba640 1 -- 172.21.15.71:0/3451307064 shutdown_connections 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c1ecba640 1 -- 172.21.15.71:0/3451307064 >> 172.21.15.71:0/3451307064 conn(0x7f6c180ad2f0 msgr2=0x7f6c180ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c1ecba640 1 -- 172.21.15.71:0/3451307064 shutdown_connections 2024-04-16T15:41:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.711+0000 7f6c1ecba640 1 -- 172.21.15.71:0/3451307064 wait complete. 2024-04-16T15:41:08.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.712+0000 7f6c1ecba640 1 Processor -- start 2024-04-16T15:41:08.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.712+0000 7f6c1ecba640 1 -- start start 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1ecba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c00004800 0x7f6c180c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1ecba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c180c5750 0x7f6c181ef130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1ecba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c181ef670 0x7f6c180af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1ecba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6c180c6480 con 0x7f6c181ef670 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1ecba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18159c60 con 0x7f6c180c5750 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1ecba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6c180c6600 con 0x7f6c00004800 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1d257640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c181ef670 0x7f6c180af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1d257640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c181ef670 0x7f6c180af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57448/0 (socket says 172.21.15.71:57448) 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.713+0000 7f6c1d257640 1 -- 172.21.15.71:0/1666566764 learned_addr learned my addr 172.21.15.71:0/1666566764 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c1d257640 1 -- 172.21.15.71:0/1666566764 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c00004800 msgr2=0x7f6c180c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c1d257640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c00004800 0x7f6c180c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c1d257640 1 -- 172.21.15.71:0/1666566764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c180c5750 msgr2=0x7f6c181ef130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c1d257640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c180c5750 0x7f6c181ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c1d257640 1 -- 172.21.15.71:0/1666566764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c180b14d0 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c1d257640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c181ef670 0x7f6c180af750 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6c10001eb0 tx=0x7f6c1000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c0d7fa640 1 -- 172.21.15.71:0/1666566764 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10019070 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c0d7fa640 1 -- 172.21.15.71:0/1666566764 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1000c5b0 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c18150390 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c0d7fa640 1 -- 172.21.15.71:0/1666566764 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1000c9c0 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.714+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c180aef50 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.715+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c18074060 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.716+0000 7f6c0d7fa640 1 -- 172.21.15.71:0/1666566764 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c1001c070 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.716+0000 7f6c0d7fa640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f6c0406c8f0 0x7f6c0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.716+0000 7f6c0d7fa640 1 -- 172.21.15.71:0/1666566764 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f6c1008b340 con 0x7f6c181ef670 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.716+0000 7f6c1ca56640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f6c0406c8f0 0x7f6c0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.716+0000 7f6c1ca56640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f6c0406c8f0 0x7f6c0406edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c140070b0 tx=0x7f6c14007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:08.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.720+0000 7f6c0d7fa640 1 -- 172.21.15.71:0/1666566764 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c10059350 con 0x7f6c181ef670 2024-04-16T15:41:08.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.939+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c1806e4c0 con 0x7f6c181ef670 2024-04-16T15:41:08.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.939+0000 7f6c0d7fa640 1 -- 172.21.15.71:0/1666566764 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f6c10058c50 con 0x7f6c181ef670 2024-04-16T15:41:08.938 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:08.938 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:40.120873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:41:08.939 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-04-16T15:41:08.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f6c0406c8f0 msgr2=0x7f6c0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f6c0406c8f0 0x7f6c0406edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c140070b0 tx=0x7f6c14007040 comp rx=0 tx=0).stop 2024-04-16T15:41:08.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c181ef670 msgr2=0x7f6c180af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:08.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c181ef670 0x7f6c180af750 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6c10001eb0 tx=0x7f6c1000eb90 comp rx=0 tx=0).stop 2024-04-16T15:41:08.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 shutdown_connections 2024-04-16T15:41:08.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f6c0406c8f0 0x7f6c0406edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c181ef670 0x7f6c180af750 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c180c5750 0x7f6c181ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 --2- 172.21.15.71:0/1666566764 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c00004800 0x7f6c180c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:08.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 >> 172.21.15.71:0/1666566764 conn(0x7f6c180ad2f0 msgr2=0x7f6c180aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:08.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 shutdown_connections 2024-04-16T15:41:08.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:08.941+0000 7f6c1ecba640 1 -- 172.21.15.71:0/1666566764 wait complete. 2024-04-16T15:41:08.951 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:09.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.056+0000 7f6240f59640 1 Processor -- start 2024-04-16T15:41:09.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.056+0000 7f6240f59640 1 -- start start 2024-04-16T15:41:09.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.056+0000 7f6240f59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c14f780 0x7f623c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.056+0000 7f6240f59640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f623c0c35f0 0x7f623c072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.056+0000 7f6240f59640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f623c072b90 0x7f623c155990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f6240f59640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f623c152060 con 0x7f623c072b90 2024-04-16T15:41:09.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f6240f59640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f623c071a30 con 0x7f623c0c35f0 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f6240f59640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f623c071d10 con 0x7f623c14f780 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623ad9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f623c072b90 0x7f623c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623ad9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f623c072b90 0x7f623c155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57454/0 (socket says 172.21.15.71:57454) 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623ad9d640 1 -- 172.21.15.71:0/2733417329 learned_addr learned my addr 172.21.15.71:0/2733417329 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623a59c640 1 --2- 172.21.15.71:0/2733417329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c14f780 0x7f623c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f6231d9b640 1 --2- 172.21.15.71:0/2733417329 >> v2:172.21.15.122:3300/0 conn(0x7f623c0c35f0 0x7f623c072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623ad9d640 1 -- 172.21.15.71:0/2733417329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c14f780 msgr2=0x7f623c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623ad9d640 1 --2- 172.21.15.71:0/2733417329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c14f780 0x7f623c06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623ad9d640 1 -- 172.21.15.71:0/2733417329 >> v2:172.21.15.122:3300/0 conn(0x7f623c0c35f0 msgr2=0x7f623c072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623ad9d640 1 --2- 172.21.15.71:0/2733417329 >> v2:172.21.15.122:3300/0 conn(0x7f623c0c35f0 0x7f623c072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f623ad9d640 1 -- 172.21.15.71:0/2733417329 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c071ff0 con 0x7f623c072b90 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.057+0000 7f6231d9b640 1 --2- 172.21.15.71:0/2733417329 >> v2:172.21.15.122:3300/0 conn(0x7f623c0c35f0 0x7f623c072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:09.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f623ad9d640 1 --2- 172.21.15.71:0/2733417329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c072b90 0x7f623c155990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f623c074670 tx=0x7f623402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b155ffce463b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6239d9b640 1 -- 172.21.15.71:0/2733417329 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234034070 con 0x7f623c072b90 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6239d9b640 1 -- 172.21.15.71:0/2733417329 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623402ee10 con 0x7f623c072b90 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6239d9b640 1 -- 172.21.15.71:0/2733417329 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234038620 con 0x7f623c072b90 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 -- 172.21.15.71:0/2733417329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c072b90 msgr2=0x7f623c155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 --2- 172.21.15.71:0/2733417329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c072b90 0x7f623c155990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f623c074670 tx=0x7f623402e060 comp rx=0 tx=0).stop 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 -- 172.21.15.71:0/2733417329 shutdown_connections 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 --2- 172.21.15.71:0/2733417329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c072b90 0x7f623c155990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 --2- 172.21.15.71:0/2733417329 >> v2:172.21.15.122:3300/0 conn(0x7f623c0c35f0 0x7f623c072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 --2- 172.21.15.71:0/2733417329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c14f780 0x7f623c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 -- 172.21.15.71:0/2733417329 >> 172.21.15.71:0/2733417329 conn(0x7f623c0c3e10 msgr2=0x7f623c0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:09.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 -- 172.21.15.71:0/2733417329 shutdown_connections 2024-04-16T15:41:09.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.058+0000 7f6240f59640 1 -- 172.21.15.71:0/2733417329 wait complete. 2024-04-16T15:41:09.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.060+0000 7f6240f59640 1 Processor -- start 2024-04-16T15:41:09.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.060+0000 7f6240f59640 1 -- start start 2024-04-16T15:41:09.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.060+0000 7f6240f59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c072b90 0x7f623c1ea550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.060+0000 7f6240f59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c0c35f0 0x7f623c0ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.060+0000 7f6240f59640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c14f780 0x7f623c14c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.060+0000 7f6240f59640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f623c06fe70 con 0x7f623c14f780 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.060+0000 7f6240f59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f623c14fb60 con 0x7f623c072b90 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.060+0000 7f6240f59640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f623c0bd300 con 0x7f623c0c35f0 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c14f780 0x7f623c14c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c14f780 0x7f623c14c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57470/0 (socket says 172.21.15.71:57470) 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 -- 172.21.15.71:0/2755854795 learned_addr learned my addr 172.21.15.71:0/2755854795 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623a59c640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c072b90 0x7f623c1ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f6231d9b640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c0c35f0 0x7f623c0ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 -- 172.21.15.71:0/2755854795 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c0c35f0 msgr2=0x7f623c0ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c0c35f0 0x7f623c0ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 -- 172.21.15.71:0/2755854795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c072b90 msgr2=0x7f623c1ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c072b90 0x7f623c1ea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 -- 172.21.15.71:0/2755854795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c0acd70 con 0x7f623c14f780 2024-04-16T15:41:09.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f623ad9d640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c14f780 0x7f623c14c7f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f623403a040 tx=0x7f6234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f62337fe640 1 -- 172.21.15.71:0/2755854795 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234048070 con 0x7f623c14f780 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f62337fe640 1 -- 172.21.15.71:0/2755854795 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6234041630 con 0x7f623c14f780 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c1eafb0 con 0x7f623c14f780 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f62337fe640 1 -- 172.21.15.71:0/2755854795 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234041a40 con 0x7f623c14f780 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.061+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c0ac9b0 con 0x7f623c14f780 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.062+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623c068d20 con 0x7f623c14f780 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.063+0000 7f62337fe640 1 -- 172.21.15.71:0/2755854795 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6234041be0 con 0x7f623c14f780 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.063+0000 7f62337fe640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f621c06c8f0 0x7f621c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.063+0000 7f62337fe640 1 -- 172.21.15.71:0/2755854795 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f62340afff0 con 0x7f623c14f780 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.063+0000 7f623a59c640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f621c06c8f0 0x7f621c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.063+0000 7f623a59c640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f621c06c8f0 0x7f621c06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f622c006fd0 tx=0x7f622c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.067+0000 7f62337fe640 1 -- 172.21.15.71:0/2755854795 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f623407e090 con 0x7f623c14f780 2024-04-16T15:41:09.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.243+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f623c06d680 con 0x7f623c14f780 2024-04-16T15:41:09.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.244+0000 7f62337fe640 1 -- 172.21.15.71:0/2755854795 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18938 (secure 0 0 0) 0x7f623407d990 con 0x7f623c14f780 2024-04-16T15:41:09.243 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:09.244 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":26,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:04.856962+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.246+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f621c06c8f0 msgr2=0x7f621c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.246+0000 7f6240f59640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f621c06c8f0 0x7f621c06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f622c006fd0 tx=0x7f622c008040 comp rx=0 tx=0).stop 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.246+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c14f780 msgr2=0x7f623c14c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.246+0000 7f6240f59640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c14f780 0x7f623c14c7f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f623403a040 tx=0x7f6234004a40 comp rx=0 tx=0).stop 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.246+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 shutdown_connections 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.246+0000 7f6240f59640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f621c06c8f0 0x7f621c06edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.247+0000 7f6240f59640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c14f780 0x7f623c14c7f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.247+0000 7f6240f59640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c0c35f0 0x7f623c0ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.247+0000 7f6240f59640 1 --2- 172.21.15.71:0/2755854795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c072b90 0x7f623c1ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.247+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 >> 172.21.15.71:0/2755854795 conn(0x7f623c0c3e10 msgr2=0x7f623c151290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.247+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 shutdown_connections 2024-04-16T15:41:09.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.247+0000 7f6240f59640 1 -- 172.21.15.71:0/2755854795 wait complete. 2024-04-16T15:41:09.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T15:40:21.794043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T15:40:22.833304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T15:40:26.544818+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T15:41:09.257 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:41:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.364+0000 7f88cf286640 1 Processor -- start 2024-04-16T15:41:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.364+0000 7f88cf286640 1 -- start start 2024-04-16T15:41:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cf286640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f88c80c8c10 0x7f88c80c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cf286640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c8155d30 con 0x7f88c8153320 2024-04-16T15:41:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cf286640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c80c3d50 con 0x7f88c81564f0 2024-04-16T15:41:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cf286640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f88c80af4c0 con 0x7f88c80c8c10 2024-04-16T15:41:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cc821640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f88c80c8c10 0x7f88c80c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cd022640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f88c8153320 0x7f88c80c4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48108/0 (socket says 172.21.15.71:48108) 2024-04-16T15:41:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cd022640 1 -- 172.21.15.71:0/3003359498 learned_addr learned my addr 172.21.15.71:0/3003359498 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cc821640 1 -- 172.21.15.71:0/3003359498 >> v1:172.21.15.122:6789/0 conn(0x7f88c81564f0 legacy=0x7f88c8071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:09.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cc821640 1 -- 172.21.15.71:0/3003359498 >> v1:172.21.15.71:6789/0 conn(0x7f88c8153320 legacy=0x7f88c80c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88cc821640 1 -- 172.21.15.71:0/3003359498 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c80c4030 con 0x7f88c80c8c10 2024-04-16T15:41:09.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.365+0000 7f88bffff640 1 -- 172.21.15.71:0/3003359498 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257737140 0 0) 0x7f88c8155d30 con 0x7f88c8153320 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.366+0000 7f88cc821640 1 --2- 172.21.15.71:0/3003359498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c80c8c10 0x7f88c80c26e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f88b80097a0 tx=0x7f88b802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30b4f310acec29a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.366+0000 7f88bffff640 1 -- 172.21.15.71:0/3003359498 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8038070 con 0x7f88c80c8c10 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.366+0000 7f88bffff640 1 -- 172.21.15.71:0/3003359498 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b8031700 con 0x7f88c80c8c10 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.366+0000 7f88bffff640 1 -- 172.21.15.71:0/3003359498 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8031ad0 con 0x7f88c80c8c10 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.367+0000 7f88cf286640 1 -- 172.21.15.71:0/3003359498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c80c8c10 msgr2=0x7f88c80c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.367+0000 7f88cf286640 1 --2- 172.21.15.71:0/3003359498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c80c8c10 0x7f88c80c26e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f88b80097a0 tx=0x7f88b802b7a0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.367+0000 7f88cf286640 1 -- 172.21.15.71:0/3003359498 shutdown_connections 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.367+0000 7f88cf286640 1 --2- 172.21.15.71:0/3003359498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c80c8c10 0x7f88c80c26e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.367+0000 7f88cf286640 1 -- 172.21.15.71:0/3003359498 >> 172.21.15.71:0/3003359498 conn(0x7f88c80c5db0 msgr2=0x7f88c80c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:09.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.367+0000 7f88cf286640 1 -- 172.21.15.71:0/3003359498 shutdown_connections 2024-04-16T15:41:09.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.367+0000 7f88cf286640 1 -- 172.21.15.71:0/3003359498 wait complete. 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.368+0000 7f88cf286640 1 Processor -- start 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.368+0000 7f88cf286640 1 -- start start 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cf286640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f88c80c8c10 0x7f88c814dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cf286640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c8153320 0x7f88c8158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cf286640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c81564f0 0x7f88c81486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cf286640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8072450 con 0x7f88c8153320 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cf286640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88c80722d0 con 0x7f88c81564f0 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cf286640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f88c8156940 con 0x7f88c80c8c10 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c8153320 0x7f88c8158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c8153320 0x7f88c8158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57496/0 (socket says 172.21.15.71:57496) 2024-04-16T15:41:09.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 -- 172.21.15.71:0/1532338702 learned_addr learned my addr 172.21.15.71:0/1532338702 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cd022640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f88c80c8c10 0x7f88c814dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cd823640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c81564f0 0x7f88c81486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 -- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f88c80c8c10 msgr2=0x7f88c814dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f88c80c8c10 0x7f88c814dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 -- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c81564f0 msgr2=0x7f88c81486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c81564f0 0x7f88c81486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 -- 172.21.15.71:0/1532338702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c814c660 con 0x7f88c8153320 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cd823640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c81564f0 0x7f88c81486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88cc821640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c8153320 0x7f88c8158b70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f88b80097a0 tx=0x7f88b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88bdffb640 1 -- 172.21.15.71:0/1532338702 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8038040 con 0x7f88c8153320 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.369+0000 7f88bdffb640 1 -- 172.21.15.71:0/1532338702 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b8033a30 con 0x7f88c8153320 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.370+0000 7f88bdffb640 1 -- 172.21.15.71:0/1532338702 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8031ed0 con 0x7f88c8153320 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.370+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c814bdc0 con 0x7f88c8153320 2024-04-16T15:41:09.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.370+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c814c380 con 0x7f88c8153320 2024-04-16T15:41:09.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.371+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c80a8db0 con 0x7f88c8153320 2024-04-16T15:41:09.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.371+0000 7f88bdffb640 1 -- 172.21.15.71:0/1532338702 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88b8004030 con 0x7f88c8153320 2024-04-16T15:41:09.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.372+0000 7f88bdffb640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f88ac06c920 0x7f88ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.372+0000 7f88bdffb640 1 -- 172.21.15.71:0/1532338702 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f88b80adc20 con 0x7f88c8153320 2024-04-16T15:41:09.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.372+0000 7f88cd022640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f88ac06c920 0x7f88ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.372+0000 7f88cd022640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f88ac06c920 0x7f88ac06ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f88c4000f60 tx=0x7f88c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.376+0000 7f88bdffb640 1 -- 172.21.15.71:0/1532338702 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88b8043020 con 0x7f88c8153320 2024-04-16T15:41:09.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.550+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f88c80c7240 con 0x7f88c8153320 2024-04-16T15:41:09.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.550+0000 7f88bdffb640 1 -- 172.21.15.71:0/1532338702 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f88b8030070 con 0x7f88c8153320 2024-04-16T15:41:09.549 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:41:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f88ac06c920 msgr2=0x7f88ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f88ac06c920 0x7f88ac06ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f88c4000f60 tx=0x7f88c4009040 comp rx=0 tx=0).stop 2024-04-16T15:41:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c8153320 msgr2=0x7f88c8158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c8153320 0x7f88c8158b70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f88b80097a0 tx=0x7f88b80046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 shutdown_connections 2024-04-16T15:41:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f88ac06c920 0x7f88ac06ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c81564f0 0x7f88c81486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88c8153320 0x7f88c8158b70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 --2- 172.21.15.71:0/1532338702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f88c80c8c10 0x7f88c814dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 >> 172.21.15.71:0/1532338702 conn(0x7f88c80c5db0 msgr2=0x7f88c80a9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 shutdown_connections 2024-04-16T15:41:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.552+0000 7f88cf286640 1 -- 172.21.15.71:0/1532338702 wait complete. 2024-04-16T15:41:09.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T15:41:09.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.669+0000 7fdbbc474640 1 Processor -- start 2024-04-16T15:41:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.669+0000 7fdbbc474640 1 -- start start 2024-04-16T15:41:09.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbbc474640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdbb40ab950 0x7fdbb40aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbbc474640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fdbb40aa900 0x7fdbb415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbbc474640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb4153ba0 con 0x7fdbb4150f40 2024-04-16T15:41:09.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbbc474640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb4153900 con 0x7fdbb40ab950 2024-04-16T15:41:09.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbbc474640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb40ad3e0 con 0x7fdbb40aa900 2024-04-16T15:41:09.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbb9a0f640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdbb40ab950 0x7fdbb40aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbb9a0f640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdbb40ab950 0x7fdbb40aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57498/0 (socket says 172.21.15.71:57498) 2024-04-16T15:41:09.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbb9a0f640 1 -- 172.21.15.71:0/2517369348 learned_addr learned my addr 172.21.15.71:0/2517369348 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:09.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbbaa11640 1 --2- 172.21.15.71:0/2517369348 >> v2:172.21.15.122:3300/0 conn(0x7fdbb40aa900 0x7fdbb415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbb9a0f640 1 -- 172.21.15.71:0/2517369348 >> v2:172.21.15.122:3300/0 conn(0x7fdbb40aa900 msgr2=0x7fdbb415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:09.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbb9a0f640 1 --2- 172.21.15.71:0/2517369348 >> v2:172.21.15.122:3300/0 conn(0x7fdbb40aa900 0x7fdbb415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbb9a0f640 1 -- 172.21.15.71:0/2517369348 >> v1:172.21.15.122:6789/0 conn(0x7fdbb4150f40 legacy=0x7fdbb40accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:09.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.670+0000 7fdbb9a0f640 1 -- 172.21.15.71:0/2517369348 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb40ac4d0 con 0x7fdbb40ab950 2024-04-16T15:41:09.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.671+0000 7fdbb9a0f640 1 --2- 172.21.15.71:0/2517369348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb40ab950 0x7fdbb40aa3c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40c8370 tx=0x7fdba402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e455e54a752d2363 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.671+0000 7fdbb920e640 1 -- 172.21.15.71:0/2517369348 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4034070 con 0x7fdbb40ab950 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.671+0000 7fdbb920e640 1 -- 172.21.15.71:0/2517369348 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba4030860 con 0x7fdbb40ab950 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.671+0000 7fdbb920e640 1 -- 172.21.15.71:0/2517369348 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4030c30 con 0x7fdbb40ab950 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.672+0000 7fdbbc474640 1 -- 172.21.15.71:0/2517369348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb40ab950 msgr2=0x7fdbb40aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.672+0000 7fdbbc474640 1 --2- 172.21.15.71:0/2517369348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb40ab950 0x7fdbb40aa3c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40c8370 tx=0x7fdba402b9a0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.672+0000 7fdbbc474640 1 -- 172.21.15.71:0/2517369348 shutdown_connections 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.672+0000 7fdbbc474640 1 --2- 172.21.15.71:0/2517369348 >> v2:172.21.15.122:3300/0 conn(0x7fdbb40aa900 0x7fdbb415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.672+0000 7fdbbc474640 1 --2- 172.21.15.71:0/2517369348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb40ab950 0x7fdbb40aa3c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.672+0000 7fdbbc474640 1 -- 172.21.15.71:0/2517369348 >> 172.21.15.71:0/2517369348 conn(0x7fdbb40c3e10 msgr2=0x7fdbb40c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.672+0000 7fdbbc474640 1 -- 172.21.15.71:0/2517369348 shutdown_connections 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.672+0000 7fdbbc474640 1 -- 172.21.15.71:0/2517369348 wait complete. 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.673+0000 7fdbbc474640 1 Processor -- start 2024-04-16T15:41:09.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.673+0000 7fdbbc474640 1 -- start start 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.673+0000 7fdbbc474640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb40aa900 0x7fdbb4147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbc474640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbb40ab950 0x7fdbb4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbc474640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb4150f40 0x7fdbb41e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbc474640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb406fdd0 con 0x7fdbb4150f40 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbc474640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb40c1f20 con 0x7fdbb40aa900 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbc474640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb406ff50 con 0x7fdbb40ab950 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb4150f40 0x7fdbb41e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb4150f40 0x7fdbb41e97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57510/0 (socket says 172.21.15.71:57510) 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 -- 172.21.15.71:0/407496984 learned_addr learned my addr 172.21.15.71:0/407496984 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:09.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbba210640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb40aa900 0x7fdbb4147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbb9a0f640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbb40ab950 0x7fdbb4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 -- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbb40ab950 msgr2=0x7fdbb4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbb40ab950 0x7fdbb4152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 -- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb40aa900 msgr2=0x7fdbb4147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb40aa900 0x7fdbb4147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 -- 172.21.15.71:0/407496984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb40c6370 con 0x7fdbb4150f40 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbba210640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb40aa900 0x7fdbb4147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbaa11640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb4150f40 0x7fdbb41e97d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00d800 tx=0x7fdbac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbaaffd640 1 -- 172.21.15.71:0/407496984 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00b6c0 con 0x7fdbb4150f40 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbaaffd640 1 -- 172.21.15.71:0/407496984 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac019070 con 0x7fdbb4150f40 2024-04-16T15:41:09.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb40c6650 con 0x7fdbb4150f40 2024-04-16T15:41:09.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbaaffd640 1 -- 172.21.15.71:0/407496984 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00bac0 con 0x7fdbb4150f40 2024-04-16T15:41:09.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.674+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb4148b60 con 0x7fdbb4150f40 2024-04-16T15:41:09.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.675+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb4068be0 con 0x7fdbb4150f40 2024-04-16T15:41:09.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.676+0000 7fdbaaffd640 1 -- 172.21.15.71:0/407496984 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbac00bc60 con 0x7fdbb4150f40 2024-04-16T15:41:09.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.676+0000 7fdbaaffd640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdb9806c920 0x7fdb9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.676+0000 7fdbaaffd640 1 -- 172.21.15.71:0/407496984 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fdbac08c360 con 0x7fdbb4150f40 2024-04-16T15:41:09.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.676+0000 7fdbba210640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdb9806c920 0x7fdb9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.677+0000 7fdbba210640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdb9806c920 0x7fdb9806ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009fc0 tx=0x7fdbb0009550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.681+0000 7fdbaaffd640 1 -- 172.21.15.71:0/407496984 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbac05a400 con 0x7fdbb4150f40 2024-04-16T15:41:09.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.855+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdbb406d680 con 0x7fdbb4150f40 2024-04-16T15:41:09.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.855+0000 7fdbaaffd640 1 -- 172.21.15.71:0/407496984 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fdbac059d00 con 0x7fdbb4150f40 2024-04-16T15:41:09.854 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdb9806c920 msgr2=0x7fdb9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdb9806c920 0x7fdb9806ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009fc0 tx=0x7fdbb0009550 comp rx=0 tx=0).stop 2024-04-16T15:41:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb4150f40 msgr2=0x7fdbb41e97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb4150f40 0x7fdbb41e97d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00d800 tx=0x7fdbac00dcc0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 shutdown_connections 2024-04-16T15:41:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdb9806c920 0x7fdb9806ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbb4150f40 0x7fdbb41e97d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbb40ab950 0x7fdbb4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 --2- 172.21.15.71:0/407496984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb40aa900 0x7fdbb4147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 >> 172.21.15.71:0/407496984 conn(0x7fdbb40c3e10 msgr2=0x7fdbb414f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:09.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 shutdown_connections 2024-04-16T15:41:09.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.857+0000 7fdbbc474640 1 -- 172.21.15.71:0/407496984 wait complete. 2024-04-16T15:41:09.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T15:41:09.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.975+0000 7f91e1f3d640 1 Processor -- start 2024-04-16T15:41:09.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.975+0000 7f91e1f3d640 1 -- start start 2024-04-16T15:41:09.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91e1f3d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91dc0c8f00 0x7f91dc1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91e1f3d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91dc0ae1c0 con 0x7f91dc153250 2024-04-16T15:41:09.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91e1f3d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91dc0ae4a0 con 0x7f91dc0ac500 2024-04-16T15:41:09.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91e1f3d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0ae780 con 0x7f91dc0c8f00 2024-04-16T15:41:09.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91db7fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f91dc153250 0x7f91dc0c2880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48114/0 (socket says 172.21.15.71:48114) 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91db7fe640 1 -- 172.21.15.71:0/2478307411 learned_addr learned my addr 172.21.15.71:0/2478307411 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069688280 0 0) 0x7f91dc0ae1c0 con 0x7f91dc153250 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91daffd640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91dc0c8f00 0x7f91dc1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91c4003520 con 0x7f91dc153250 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3408887824 0 0) 0x7f91c4003520 con 0x7f91dc153250 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91dc0c8f00 msgr2=0x7f91dc1596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91da7fc640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91dc0c8f00 0x7f91dc1596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.976+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 >> v1:172.21.15.122:6789/0 conn(0x7f91dc0ac500 legacy=0x7f91dc160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc0aea60 con 0x7f91dc153250 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f91cc001670 con 0x7f91dc153250 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 >> v1:172.21.15.71:6789/0 conn(0x7f91dc153250 legacy=0x7f91dc0c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91dc0c8f00 0x7f91c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91c4003f60 0x7f91c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c4004d40 0x7f91c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0ae780 con 0x7f91c4004d40 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f91c400bb30 con 0x7f91c4003f60 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f91c400bd30 con 0x7f91dc0c8f00 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91dbfff640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c4004d40 0x7f91c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91db7fe640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91dc0c8f00 0x7f91c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91daffd640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91c4003f60 0x7f91c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91dbfff640 1 -- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91dc0c8f00 msgr2=0x7f91c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91dbfff640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91dc0c8f00 0x7f91c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91dbfff640 1 -- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91c4003f60 msgr2=0x7f91c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91dbfff640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91c4003f60 0x7f91c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91dbfff640 1 -- 172.21.15.71:0/2478307411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c4003520 con 0x7f91c4004d40 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.977+0000 7f91daffd640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91c4003f60 0x7f91c4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91dbfff640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c4004d40 0x7f91c40051c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91d4001e50 tx=0x7f91c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8338a27b69863ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4001a70 con 0x7f91c4004d40 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d4003b70 con 0x7f91c4004d40 2024-04-16T15:41:09.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91da7fc640 1 -- 172.21.15.71:0/2478307411 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d40050d0 con 0x7f91c4004d40 2024-04-16T15:41:09.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91e1f3d640 1 -- 172.21.15.71:0/2478307411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c4004d40 msgr2=0x7f91c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91e1f3d640 1 --2- 172.21.15.71:0/2478307411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c4004d40 0x7f91c40051c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91d4001e50 tx=0x7f91c4005830 comp rx=0 tx=0).stop 2024-04-16T15:41:09.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91dbfff640 1 -- 172.21.15.71:0/2478307411 reap_dead start 2024-04-16T15:41:09.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91e1f3d640 1 -- 172.21.15.71:0/2478307411 shutdown_connections 2024-04-16T15:41:09.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.978+0000 7f91e1f3d640 1 -- 172.21.15.71:0/2478307411 >> 172.21.15.71:0/2478307411 conn(0x7f91dc0ab200 msgr2=0x7f91dc0ab610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:09.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.979+0000 7f91e1f3d640 1 -- 172.21.15.71:0/2478307411 shutdown_connections 2024-04-16T15:41:09.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.979+0000 7f91e1f3d640 1 -- 172.21.15.71:0/2478307411 wait complete. 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.980+0000 7f91e1f3d640 1 Processor -- start 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.980+0000 7f91e1f3d640 1 -- start start 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91e1f3d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91c4004d40 0x7f91dc1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91e1f3d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91dc14e360 0x7f91dc14a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91e1f3d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91dc14b7b0 0x7f91dc1ee730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91e1f3d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0c3110 con 0x7f91dc14e360 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91e1f3d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0c2f90 con 0x7f91dc14b7b0 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91e1f3d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0c7350 con 0x7f91c4004d40 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91dc14e360 0x7f91dc14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91dc14e360 0x7f91dc14a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57534/0 (socket says 172.21.15.71:57534) 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 -- 172.21.15.71:0/3613569490 learned_addr learned my addr 172.21.15.71:0/3613569490 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:09.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91db7fe640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91c4004d40 0x7f91dc1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 -- 172.21.15.71:0/3613569490 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91c4004d40 msgr2=0x7f91dc1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91c4004d40 0x7f91dc1497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91dbfff640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91dc14b7b0 0x7f91dc1ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 -- 172.21.15.71:0/3613569490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91dc14b7b0 msgr2=0x7f91dc1ee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91dc14b7b0 0x7f91dc1ee730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 -- 172.21.15.71:0/3613569490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc1566a0 con 0x7f91dc14e360 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91daffd640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91dc14e360 0x7f91dc14a930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f91d0011560 tx=0x7f91d0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91c3fff640 1 -- 172.21.15.71:0/3613569490 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0015870 con 0x7f91dc14e360 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91c3fff640 1 -- 172.21.15.71:0/3613569490 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d0015a10 con 0x7f91dc14e360 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc156980 con 0x7f91dc14e360 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91c3fff640 1 -- 172.21.15.71:0/3613569490 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0015dc0 con 0x7f91dc14e360 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.981+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc148fc0 con 0x7f91dc14e360 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.982+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc068be0 con 0x7f91dc14e360 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.983+0000 7f91c3fff640 1 -- 172.21.15.71:0/3613569490 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91d000f460 con 0x7f91dc14e360 2024-04-16T15:41:09.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.983+0000 7f91c3fff640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f91c806c8f0 0x7f91c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.983+0000 7f91c3fff640 1 -- 172.21.15.71:0/3613569490 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f91d008c840 con 0x7f91dc14e360 2024-04-16T15:41:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.983+0000 7f91db7fe640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f91c806c8f0 0x7f91c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.984+0000 7f91db7fe640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f91c806c8f0 0x7f91c806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91cc001050 tx=0x7f91cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:09.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:09.988+0000 7f91c3fff640 1 -- 172.21.15.71:0/3613569490 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91d0019030 con 0x7f91dc14e360 2024-04-16T15:41:10.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.163+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f91dc0ad7a0 con 0x7f91dc14e360 2024-04-16T15:41:10.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.163+0000 7f91c3fff640 1 -- 172.21.15.71:0/3613569490 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f91d005a8e0 con 0x7f91dc14e360 2024-04-16T15:41:10.162 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-04-16T15:41:10.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f91c806c8f0 msgr2=0x7f91c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f91c806c8f0 0x7f91c806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91cc001050 tx=0x7f91cc009040 comp rx=0 tx=0).stop 2024-04-16T15:41:10.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91dc14e360 msgr2=0x7f91dc14a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91dc14e360 0x7f91dc14a930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f91d0011560 tx=0x7f91d0011a20 comp rx=0 tx=0).stop 2024-04-16T15:41:10.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 shutdown_connections 2024-04-16T15:41:10.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f91c806c8f0 0x7f91c806edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f91dc14b7b0 0x7f91dc1ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91dc14e360 0x7f91dc14a930 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 --2- 172.21.15.71:0/3613569490 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f91c4004d40 0x7f91dc1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 >> 172.21.15.71:0/3613569490 conn(0x7f91dc0ab200 msgr2=0x7f91dc1595c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:10.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 shutdown_connections 2024-04-16T15:41:10.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.165+0000 7f91e1f3d640 1 -- 172.21.15.71:0/3613569490 wait complete. 2024-04-16T15:41:10.175 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T15:41:10.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.282+0000 7facdc677640 1 Processor -- start 2024-04-16T15:41:10.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.282+0000 7facdc677640 1 -- start start 2024-04-16T15:41:10.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.283+0000 7facdc677640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7facd4155dc0 0x7facd40c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.283+0000 7facdc677640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7facd406fac0 0x7facd41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.283+0000 7facdc677640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd40c27c0 con 0x7facd40ab290 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.283+0000 7facdc677640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7facd4073f20 con 0x7facd4155dc0 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.283+0000 7facdc677640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7facd4156ab0 con 0x7facd406fac0 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.283+0000 7facda413640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7facd4155dc0 0x7facd40c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.283+0000 7facdac14640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7facd40ab290 0x7facd41527d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48118/0 (socket says 172.21.15.71:48118) 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.283+0000 7facdac14640 1 -- 172.21.15.71:0/3971919242 learned_addr learned my addr 172.21.15.71:0/3971919242 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facd9c12640 1 --2- 172.21.15.71:0/3971919242 >> v2:172.21.15.122:3300/0 conn(0x7facd406fac0 0x7facd41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facda413640 1 -- 172.21.15.71:0/3971919242 >> v2:172.21.15.122:3300/0 conn(0x7facd406fac0 msgr2=0x7facd41561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facda413640 1 --2- 172.21.15.71:0/3971919242 >> v2:172.21.15.122:3300/0 conn(0x7facd406fac0 0x7facd41561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facda413640 1 -- 172.21.15.71:0/3971919242 >> v1:172.21.15.71:6789/0 conn(0x7facd40ab290 legacy=0x7facd41527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facda413640 1 -- 172.21.15.71:0/3971919242 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd40c2aa0 con 0x7facd4155dc0 2024-04-16T15:41:10.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facd9411640 1 -- 172.21.15.71:0/3971919242 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423776212 0 0) 0x7facd40c27c0 con 0x7facd40ab290 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facda413640 1 --2- 172.21.15.71:0/3971919242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd40c32a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7facd406d2d0 tx=0x7facc402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4486efbadd4a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facd9411640 1 -- 172.21.15.71:0/3971919242 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4039070 con 0x7facd4155dc0 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facd9411640 1 -- 172.21.15.71:0/3971919242 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc402cc80 con 0x7facd4155dc0 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.284+0000 7facd9411640 1 -- 172.21.15.71:0/3971919242 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4034660 con 0x7facd4155dc0 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.285+0000 7facdc677640 1 -- 172.21.15.71:0/3971919242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 msgr2=0x7facd40c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.285+0000 7facdc677640 1 --2- 172.21.15.71:0/3971919242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd40c32a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7facd406d2d0 tx=0x7facc402bed0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.285+0000 7facdc677640 1 -- 172.21.15.71:0/3971919242 shutdown_connections 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.285+0000 7facdc677640 1 --2- 172.21.15.71:0/3971919242 >> v2:172.21.15.122:3300/0 conn(0x7facd406fac0 0x7facd41561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.285+0000 7facdc677640 1 --2- 172.21.15.71:0/3971919242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd40c32a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.285+0000 7facdc677640 1 -- 172.21.15.71:0/3971919242 >> 172.21.15.71:0/3971919242 conn(0x7facd40c3e10 msgr2=0x7facd40c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.285+0000 7facdc677640 1 -- 172.21.15.71:0/3971919242 shutdown_connections 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.285+0000 7facdc677640 1 -- 172.21.15.71:0/3971919242 wait complete. 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.286+0000 7facdc677640 1 Processor -- start 2024-04-16T15:41:10.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.286+0000 7facdc677640 1 -- start start 2024-04-16T15:41:10.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facdc677640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facd40ab290 0x7facd41ed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facdc677640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd4154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facdc677640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facd4154dd0 0x7facd40c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facdc677640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7facd41566e0 con 0x7facd4155dc0 2024-04-16T15:41:10.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facdc677640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7facd40c18b0 con 0x7facd4154dd0 2024-04-16T15:41:10.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facdc677640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7facd4156860 con 0x7facd40ab290 2024-04-16T15:41:10.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd4154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd4154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57558/0 (socket says 172.21.15.71:57558) 2024-04-16T15:41:10.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 -- 172.21.15.71:0/681281004 learned_addr learned my addr 172.21.15.71:0/681281004 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:10.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facda413640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facd40ab290 0x7facd41ed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facdac14640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facd4154dd0 0x7facd40c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 -- 172.21.15.71:0/681281004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facd40ab290 msgr2=0x7facd41ed4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facd40ab290 0x7facd41ed4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 -- 172.21.15.71:0/681281004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facd4154dd0 msgr2=0x7facd40c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facd4154dd0 0x7facd40c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 -- 172.21.15.71:0/681281004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd41eda30 con 0x7facd4155dc0 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.287+0000 7facd9c12640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd4154440 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7facc800d7d0 tx=0x7facc800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.288+0000 7facd2ffd640 1 -- 172.21.15.71:0/681281004 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc800b6c0 con 0x7facd4155dc0 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.288+0000 7facd2ffd640 1 -- 172.21.15.71:0/681281004 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc8019070 con 0x7facd4155dc0 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.288+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd4155360 con 0x7facd4155dc0 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.288+0000 7facd2ffd640 1 -- 172.21.15.71:0/681281004 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc800bac0 con 0x7facd4155dc0 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.288+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd4154980 con 0x7facd4155dc0 2024-04-16T15:41:10.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.289+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd4155bf0 con 0x7facd4155dc0 2024-04-16T15:41:10.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.289+0000 7facd2ffd640 1 -- 172.21.15.71:0/681281004 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facc800bc60 con 0x7facd4155dc0 2024-04-16T15:41:10.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.290+0000 7facd2ffd640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facc006c8f0 0x7facc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.290+0000 7facd2ffd640 1 -- 172.21.15.71:0/681281004 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7facc808c360 con 0x7facd4155dc0 2024-04-16T15:41:10.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.290+0000 7facda413640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facc006c8f0 0x7facc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.290+0000 7facda413640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facc006c8f0 0x7facc006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7facc4036000 tx=0x7facc4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.294+0000 7facd2ffd640 1 -- 172.21.15.71:0/681281004 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7facc801c360 con 0x7facd4155dc0 2024-04-16T15:41:10.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.468+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7facd41490d0 con 0x7facd4155dc0 2024-04-16T15:41:10.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.468+0000 7facd2ffd640 1 -- 172.21.15.71:0/681281004 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7facd41490d0 con 0x7facd4155dc0 2024-04-16T15:41:10.467 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facc006c8f0 msgr2=0x7facc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facc006c8f0 0x7facc006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7facc4036000 tx=0x7facc4030000 comp rx=0 tx=0).stop 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 msgr2=0x7facd4154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd4154440 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7facc800d7d0 tx=0x7facc800dc90 comp rx=0 tx=0).stop 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 shutdown_connections 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7facc006c8f0 0x7facc006edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facd4154dd0 0x7facd40c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd4155dc0 0x7facd4154440 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 --2- 172.21.15.71:0/681281004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facd40ab290 0x7facd41ed4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 >> 172.21.15.71:0/681281004 conn(0x7facd40c3e10 msgr2=0x7facd413cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:10.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 shutdown_connections 2024-04-16T15:41:10.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.470+0000 7facdc677640 1 -- 172.21.15.71:0/681281004 wait complete. 2024-04-16T15:41:10.480 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:41:10.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.584+0000 7f525158f640 1 Processor -- start 2024-04-16T15:41:10.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.584+0000 7f525158f640 1 -- start start 2024-04-16T15:41:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f525158f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f524c159520 0x7f524c159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f525158f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f524c0c78d0 0x7f524c06df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f525158f640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f524c0ae4b0 con 0x7f524c06e470 2024-04-16T15:41:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f525158f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f524c0c9540 con 0x7f524c0c78d0 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f525158f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f524c0c6940 con 0x7f524c159520 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524b7fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f524c06e470 0x7f524c0ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48120/0 (socket says 172.21.15.71:48120) 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524b7fe640 1 -- 172.21.15.71:0/4026119052 learned_addr learned my addr 172.21.15.71:0/4026119052 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739565041 0 0) 0x7f524c0ae4b0 con 0x7f524c06e470 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524affd640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f524c159520 0x7f524c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5238003520 con 0x7f524c06e470 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f52427fc640 1 --2- 172.21.15.71:0/4026119052 >> v2:172.21.15.122:3300/0 conn(0x7f524c0c78d0 0x7f524c06df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3360688726 0 0) 0x7f5238003520 con 0x7f524c06e470 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f524c159520 msgr2=0x7f524c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f524c159520 0x7f524c159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 >> v2:172.21.15.122:3300/0 conn(0x7f524c0c78d0 msgr2=0x7f524c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 --2- 172.21.15.71:0/4026119052 >> v2:172.21.15.122:3300/0 conn(0x7f524c0c78d0 0x7f524c06df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c15a030 con 0x7f524c06e470 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f52440019f0 con 0x7f524c06e470 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.585+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 >> v1:172.21.15.71:6789/0 conn(0x7f524c06e470 legacy=0x7f524c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524a7fc640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f524c159520 0x7f52380039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524a7fc640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5238003f50 0x7f5238004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524a7fc640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5238004d40 0x7f52380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f524c0c9540 con 0x7f5238004d40 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f524c0c6940 con 0x7f5238003f50 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f523800bac0 con 0x7f524c159520 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524b7fe640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5238004d40 0x7f52380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f52427fc640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5238003f50 0x7f5238004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524affd640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f524c159520 0x7f52380039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524b7fe640 1 -- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f524c159520 msgr2=0x7f52380039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524b7fe640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f524c159520 0x7f52380039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524b7fe640 1 -- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5238003f50 msgr2=0x7f5238004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524b7fe640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5238003f50 0x7f5238004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.586+0000 7f524b7fe640 1 -- 172.21.15.71:0/4026119052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5238003520 con 0x7f5238004d40 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f524b7fe640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5238004d40 0x7f52380051c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f524400a870 tx=0x7f5238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77fd12ab2fee4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524400a4c0 con 0x7f5238004d40 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5244034630 con 0x7f5238004d40 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f524a7fc640 1 -- 172.21.15.71:0/4026119052 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244034a20 con 0x7f5238004d40 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f525158f640 1 -- 172.21.15.71:0/4026119052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5238004d40 msgr2=0x7f52380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f525158f640 1 --2- 172.21.15.71:0/4026119052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5238004d40 0x7f52380051c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f524400a870 tx=0x7f5238005830 comp rx=0 tx=0).stop 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f524b7fe640 1 -- 172.21.15.71:0/4026119052 reap_dead start 2024-04-16T15:41:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f525158f640 1 -- 172.21.15.71:0/4026119052 shutdown_connections 2024-04-16T15:41:10.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f525158f640 1 -- 172.21.15.71:0/4026119052 >> 172.21.15.71:0/4026119052 conn(0x7f524c0cb820 msgr2=0x7f524c0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:10.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f525158f640 1 -- 172.21.15.71:0/4026119052 shutdown_connections 2024-04-16T15:41:10.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.587+0000 7f525158f640 1 -- 172.21.15.71:0/4026119052 wait complete. 2024-04-16T15:41:10.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.588+0000 7f525158f640 1 Processor -- start 2024-04-16T15:41:10.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f525158f640 1 -- start start 2024-04-16T15:41:10.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f525158f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5238004d40 0x7f524c1ec610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f525158f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f524c1561f0 0x7f524c0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f525158f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c153b30 0x7f524c1ef030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f525158f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f524c0c9d50 con 0x7f524c1561f0 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f525158f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f524c0c9a70 con 0x7f524c153b30 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f525158f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f524c0b0cf0 con 0x7f5238004d40 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f52427fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f524c1561f0 0x7f524c0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f52427fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f524c1561f0 0x7f524c0ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57574/0 (socket says 172.21.15.71:57574) 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.589+0000 7f52427fc640 1 -- 172.21.15.71:0/938835096 learned_addr learned my addr 172.21.15.71:0/938835096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f524affd640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5238004d40 0x7f524c1ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f52427fc640 1 -- 172.21.15.71:0/938835096 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5238004d40 msgr2=0x7f524c1ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f524b7fe640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c153b30 0x7f524c1ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f52427fc640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5238004d40 0x7f524c1ec610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f52427fc640 1 -- 172.21.15.71:0/938835096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c153b30 msgr2=0x7f524c1ef030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f52427fc640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c153b30 0x7f524c1ef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f52427fc640 1 -- 172.21.15.71:0/938835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c155540 con 0x7f524c1561f0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f52427fc640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f524c1561f0 0x7f524c0ac880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5234011960 tx=0x7f5234011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f5243fff640 1 -- 172.21.15.71:0/938835096 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523400fa90 con 0x7f524c1561f0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f5243fff640 1 -- 172.21.15.71:0/938835096 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523400fc30 con 0x7f524c1561f0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c155820 con 0x7f524c1561f0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f5243fff640 1 -- 172.21.15.71:0/938835096 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234014440 con 0x7f524c1561f0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.590+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c0acdc0 con 0x7f524c1561f0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.591+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524c074060 con 0x7f524c1561f0 2024-04-16T15:41:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.592+0000 7f5243fff640 1 -- 172.21.15.71:0/938835096 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5234012040 con 0x7f524c1561f0 2024-04-16T15:41:10.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.592+0000 7f5243fff640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f522c06c8f0 0x7f522c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.592+0000 7f5243fff640 1 -- 172.21.15.71:0/938835096 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f523408c510 con 0x7f524c1561f0 2024-04-16T15:41:10.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.592+0000 7f524affd640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f522c06c8f0 0x7f522c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.592+0000 7f524affd640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f522c06c8f0 0x7f522c06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f524c06e2f0 tx=0x7f523c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.597+0000 7f5243fff640 1 -- 172.21.15.71:0/938835096 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5234019030 con 0x7f524c1561f0 2024-04-16T15:41:10.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.806+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f524c0c6280 con 0x7f524c1561f0 2024-04-16T15:41:10.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.807+0000 7f5243fff640 1 -- 172.21.15.71:0/938835096 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f523405a5b0 con 0x7f524c1561f0 2024-04-16T15:41:10.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:10.806 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:40.120873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:41:10.807 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f522c06c8f0 msgr2=0x7f522c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f522c06c8f0 0x7f522c06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f524c06e2f0 tx=0x7f523c008040 comp rx=0 tx=0).stop 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f524c1561f0 msgr2=0x7f524c0ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f524c1561f0 0x7f524c0ac880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5234011960 tx=0x7f5234011e20 comp rx=0 tx=0).stop 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 shutdown_connections 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f522c06c8f0 0x7f522c06edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c153b30 0x7f524c1ef030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f524c1561f0 0x7f524c0ac880 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 --2- 172.21.15.71:0/938835096 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5238004d40 0x7f524c1ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 >> 172.21.15.71:0/938835096 conn(0x7f524c0cb820 msgr2=0x7f524c151800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 shutdown_connections 2024-04-16T15:41:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.809+0000 7f525158f640 1 -- 172.21.15.71:0/938835096 wait complete. 2024-04-16T15:41:10.819 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:10.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.923+0000 7fec1a442640 1 Processor -- start 2024-04-16T15:41:10.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.923+0000 7fec1a442640 1 -- start start 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec1a442640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec14155dc0 0x7fec140c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec1a442640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fec1406fac0 0x7fec141561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec1a442640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fec140ab290 0x7fec141527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec1a442640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fec14073f20 con 0x7fec140ab290 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec1a442640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fec14156ab0 con 0x7fec1406fac0 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec1a442640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fec140c27c0 con 0x7fec14155dc0 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec189df640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fec140ab290 0x7fec141527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec189df640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fec140ab290 0x7fec141527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57580/0 (socket says 172.21.15.71:57580) 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec189df640 1 -- 172.21.15.71:0/282700477 learned_addr learned my addr 172.21.15.71:0/282700477 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec189df640 1 -- 172.21.15.71:0/282700477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec14155dc0 msgr2=0x7fec140c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec189df640 1 --2- 172.21.15.71:0/282700477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec14155dc0 0x7fec140c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec189df640 1 -- 172.21.15.71:0/282700477 >> v2:172.21.15.122:3300/0 conn(0x7fec1406fac0 msgr2=0x7fec141561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec189df640 1 --2- 172.21.15.71:0/282700477 >> v2:172.21.15.122:3300/0 conn(0x7fec1406fac0 0x7fec141561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.924+0000 7fec189df640 1 -- 172.21.15.71:0/282700477 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec140c2aa0 con 0x7fec140ab290 2024-04-16T15:41:10.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.925+0000 7fec189df640 1 --2- 172.21.15.71:0/282700477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141527d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec140746b0 tx=0x7fec0c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753ff8fa20e7a6c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.925+0000 7fec12ffd640 1 -- 172.21.15.71:0/282700477 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c02ec80 con 0x7fec140ab290 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.925+0000 7fec12ffd640 1 -- 172.21.15.71:0/282700477 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec0c02ee20 con 0x7fec140ab290 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.925+0000 7fec12ffd640 1 -- 172.21.15.71:0/282700477 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c036610 con 0x7fec140ab290 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 -- 172.21.15.71:0/282700477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 msgr2=0x7fec141527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 --2- 172.21.15.71:0/282700477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141527d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec140746b0 tx=0x7fec0c02e0e0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 -- 172.21.15.71:0/282700477 shutdown_connections 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 --2- 172.21.15.71:0/282700477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141527d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 --2- 172.21.15.71:0/282700477 >> v2:172.21.15.122:3300/0 conn(0x7fec1406fac0 0x7fec141561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 --2- 172.21.15.71:0/282700477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec14155dc0 0x7fec140c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 -- 172.21.15.71:0/282700477 >> 172.21.15.71:0/282700477 conn(0x7fec140c3e10 msgr2=0x7fec140c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:10.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 -- 172.21.15.71:0/282700477 shutdown_connections 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.926+0000 7fec1a442640 1 -- 172.21.15.71:0/282700477 wait complete. 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.927+0000 7fec1a442640 1 Processor -- start 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.927+0000 7fec1a442640 1 -- start start 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec1a442640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec1a442640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec14155dc0 0x7fec14141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec1a442640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec140c5910 0x7fec14142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec1a442640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fec141566e0 con 0x7fec140ab290 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec1a442640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec140c18b0 con 0x7fec14155dc0 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec1a442640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fec14156860 con 0x7fec140c5910 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec13fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec13fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57586/0 (socket says 172.21.15.71:57586) 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec13fff640 1 -- 172.21.15.71:0/1067436683 learned_addr learned my addr 172.21.15.71:0/1067436683 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:10.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec137fe640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec14155dc0 0x7fec14141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec189df640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec140c5910 0x7fec14142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec13fff640 1 -- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec140c5910 msgr2=0x7fec14142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec13fff640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec140c5910 0x7fec14142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec13fff640 1 -- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec14155dc0 msgr2=0x7fec14141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec13fff640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec14155dc0 0x7fec14141af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec13fff640 1 -- 172.21.15.71:0/1067436683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec140c5e10 con 0x7fec140ab290 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.928+0000 7fec137fe640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec14155dc0 0x7fec14141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.929+0000 7fec13fff640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141453e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec0400b440 tx=0x7fec0400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.929+0000 7fec10ff9640 1 -- 172.21.15.71:0/1067436683 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec04015020 con 0x7fec140ab290 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.929+0000 7fec10ff9640 1 -- 172.21.15.71:0/1067436683 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec04011890 con 0x7fec140ab290 2024-04-16T15:41:10.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.929+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec140c60f0 con 0x7fec140ab290 2024-04-16T15:41:10.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.929+0000 7fec10ff9640 1 -- 172.21.15.71:0/1067436683 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec04011c80 con 0x7fec140ab290 2024-04-16T15:41:10.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.929+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec141e9650 con 0x7fec140ab290 2024-04-16T15:41:10.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.930+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec14068d20 con 0x7fec140ab290 2024-04-16T15:41:10.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.930+0000 7fec10ff9640 1 -- 172.21.15.71:0/1067436683 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec04018020 con 0x7fec140ab290 2024-04-16T15:41:10.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.931+0000 7fec10ff9640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec0006c8f0 0x7fec0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:10.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.931+0000 7fec137fe640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec0006c8f0 0x7fec0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:10.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.931+0000 7fec10ff9640 1 -- 172.21.15.71:0/1067436683 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fec0408c090 con 0x7fec140ab290 2024-04-16T15:41:10.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.931+0000 7fec137fe640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec0006c8f0 0x7fec0006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fec141464d0 tx=0x7fec08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:10.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:10.935+0000 7fec10ff9640 1 -- 172.21.15.71:0/1067436683 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec0401f020 con 0x7fec140ab290 2024-04-16T15:41:11.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.112+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec14146c20 con 0x7fec140ab290 2024-04-16T15:41:11.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.114+0000 7fec10ff9640 1 -- 172.21.15.71:0/1067436683 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18938 (secure 0 0 0) 0x7fec0400d040 con 0x7fec140ab290 2024-04-16T15:41:11.113 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:11.113 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":26,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:04.856962+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.115+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec0006c8f0 msgr2=0x7fec0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.115+0000 7fec1a442640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec0006c8f0 0x7fec0006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fec141464d0 tx=0x7fec08008040 comp rx=0 tx=0).stop 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.115+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 msgr2=0x7fec141453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.115+0000 7fec1a442640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141453e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec0400b440 tx=0x7fec0400b900 comp rx=0 tx=0).stop 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.116+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 shutdown_connections 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.116+0000 7fec1a442640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fec0006c8f0 0x7fec0006edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.116+0000 7fec1a442640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fec140c5910 0x7fec14142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.116+0000 7fec1a442640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec14155dc0 0x7fec14141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.116+0000 7fec1a442640 1 --2- 172.21.15.71:0/1067436683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec140ab290 0x7fec141453e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.116+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 >> 172.21.15.71:0/1067436683 conn(0x7fec140c3e10 msgr2=0x7fec140ac230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:11.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.116+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 shutdown_connections 2024-04-16T15:41:11.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.116+0000 7fec1a442640 1 -- 172.21.15.71:0/1067436683 wait complete. 2024-04-16T15:41:11.125 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-16T15:41:11.126 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:41:11.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.230+0000 7f336411b640 1 Processor -- start 2024-04-16T15:41:11.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.231+0000 7f336411b640 1 -- start start 2024-04-16T15:41:11.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.231+0000 7f336411b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c0c94c0 0x7f335c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.231+0000 7f336411b640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f335c06e820 con 0x7f335c0c99d0 2024-04-16T15:41:11.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.231+0000 7f336411b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f335c159fb0 con 0x7f335c153450 2024-04-16T15:41:11.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.231+0000 7f336411b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f335c0c8150 con 0x7f335c0c94c0 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.231+0000 7f33616b6640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f335c0c99d0 0x7f335c155db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48130/0 (socket says 172.21.15.71:48130) 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.231+0000 7f33616b6640 1 -- 172.21.15.71:0/2114797139 learned_addr learned my addr 172.21.15.71:0/2114797139 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f33626b8640 1 --2- 172.21.15.71:0/2114797139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c0c94c0 0x7f335c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f3360eb5640 1 -- 172.21.15.71:0/2114797139 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1036191192 0 0) 0x7f335c06e820 con 0x7f335c0c99d0 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f3360eb5640 1 -- 172.21.15.71:0/2114797139 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3344003520 con 0x7f335c0c99d0 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f3360eb5640 1 -- 172.21.15.71:0/2114797139 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1062104453 0 0) 0x7f335c159fb0 con 0x7f335c153450 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f3360eb5640 1 -- 172.21.15.71:0/2114797139 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f335c06e820 con 0x7f335c153450 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f3360eb5640 1 -- 172.21.15.71:0/2114797139 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3539390826 0 0) 0x7f3344003520 con 0x7f335c0c99d0 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f33626b8640 1 -- 172.21.15.71:0/2114797139 >> v1:172.21.15.122:6789/0 conn(0x7f335c153450 legacy=0x7f335c0c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f33626b8640 1 -- 172.21.15.71:0/2114797139 >> v1:172.21.15.71:6789/0 conn(0x7f335c0c99d0 legacy=0x7f335c155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.232+0000 7f33626b8640 1 -- 172.21.15.71:0/2114797139 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c0c8430 con 0x7f335c0c94c0 2024-04-16T15:41:11.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.233+0000 7f33626b8640 1 --2- 172.21.15.71:0/2114797139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c0c94c0 0x7f335c1564c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f33540097d0 tx=0x7f335402b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d2b77d3ace38573 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.233+0000 7f3360eb5640 1 -- 172.21.15.71:0/2114797139 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354037070 con 0x7f335c0c94c0 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.233+0000 7f3360eb5640 1 -- 172.21.15.71:0/2114797139 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3354030510 con 0x7f335c0c94c0 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.233+0000 7f3360eb5640 1 -- 172.21.15.71:0/2114797139 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33500019f0 con 0x7f335c0c94c0 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.234+0000 7f336411b640 1 -- 172.21.15.71:0/2114797139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c0c94c0 msgr2=0x7f335c1564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.234+0000 7f336411b640 1 --2- 172.21.15.71:0/2114797139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c0c94c0 0x7f335c1564c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f33540097d0 tx=0x7f335402b6e0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.234+0000 7f336411b640 1 -- 172.21.15.71:0/2114797139 shutdown_connections 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.234+0000 7f336411b640 1 --2- 172.21.15.71:0/2114797139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c0c94c0 0x7f335c1564c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.234+0000 7f336411b640 1 -- 172.21.15.71:0/2114797139 >> 172.21.15.71:0/2114797139 conn(0x7f335c0ad2f0 msgr2=0x7f335c0ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.234+0000 7f336411b640 1 -- 172.21.15.71:0/2114797139 shutdown_connections 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.234+0000 7f336411b640 1 -- 172.21.15.71:0/2114797139 wait complete. 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f336411b640 1 Processor -- start 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f336411b640 1 -- start start 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f336411b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c0c94c0 0x7f335c14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f336411b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c0c99d0 0x7f335c1ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f336411b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c153450 0x7f335c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f336411b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f335c0c6650 con 0x7f335c0c94c0 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f336411b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f335c159c60 con 0x7f335c0c99d0 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f336411b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f335c0c67d0 con 0x7f335c153450 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f3361eb7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c0c94c0 0x7f335c14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f3361eb7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c0c94c0 0x7f335c14d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57596/0 (socket says 172.21.15.71:57596) 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.235+0000 7f3361eb7640 1 -- 172.21.15.71:0/75540428 learned_addr learned my addr 172.21.15.71:0/75540428 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f33626b8640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c153450 0x7f335c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f33616b6640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c0c99d0 0x7f335c1ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f3361eb7640 1 -- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c153450 msgr2=0x7f335c158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f3361eb7640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c153450 0x7f335c158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f3361eb7640 1 -- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c0c99d0 msgr2=0x7f335c1ebdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f3361eb7640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c0c99d0 0x7f335c1ebdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f3361eb7640 1 -- 172.21.15.71:0/75540428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c0b0060 con 0x7f335c0c94c0 2024-04-16T15:41:11.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f33626b8640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c153450 0x7f335c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f3361eb7640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c0c94c0 0x7f335c14d6f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c4c0 tx=0x7f334c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f335a7fc640 1 -- 172.21.15.71:0/75540428 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c0052d0 con 0x7f335c0c94c0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f335a7fc640 1 -- 172.21.15.71:0/75540428 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f334c005470 con 0x7f335c0c94c0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335c0b0340 con 0x7f335c0c94c0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f335a7fc640 1 -- 172.21.15.71:0/75540428 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c012870 con 0x7f335c0c94c0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.236+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335c1ee9c0 con 0x7f335c0c94c0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.237+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335c074060 con 0x7f335c0c94c0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.238+0000 7f335a7fc640 1 -- 172.21.15.71:0/75540428 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f334c012a10 con 0x7f335c0c94c0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.238+0000 7f335a7fc640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f334806c920 0x7f334806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.238+0000 7f335a7fc640 1 -- 172.21.15.71:0/75540428 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f334c08bc80 con 0x7f335c0c94c0 2024-04-16T15:41:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.238+0000 7f33616b6640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f334806c920 0x7f334806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.238+0000 7f33616b6640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f334806c920 0x7f334806ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f335c0c5df0 tx=0x7f3350009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.242+0000 7f335a7fc640 1 -- 172.21.15.71:0/75540428 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f334c059c90 con 0x7f335c0c94c0 2024-04-16T15:41:11.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.454+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f335c14d4c0 con 0x7f335c0c94c0 2024-04-16T15:41:11.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.454+0000 7f335a7fc640 1 -- 172.21.15.71:0/75540428 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f335c14d4c0 con 0x7f335c0c94c0 2024-04-16T15:41:11.453 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:11.454 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:40.120873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:41:11.454 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-04-16T15:41:11.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f334806c920 msgr2=0x7f334806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f334806c920 0x7f334806ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f335c0c5df0 tx=0x7f3350009040 comp rx=0 tx=0).stop 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c0c94c0 msgr2=0x7f335c14d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c0c94c0 0x7f335c14d6f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c4c0 tx=0x7f334c00c980 comp rx=0 tx=0).stop 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 shutdown_connections 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f334806c920 0x7f334806ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f335c153450 0x7f335c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f335c0c99d0 0x7f335c1ebdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 --2- 172.21.15.71:0/75540428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c0c94c0 0x7f335c14d6f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 >> 172.21.15.71:0/75540428 conn(0x7f335c0ad2f0 msgr2=0x7f335c0ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 shutdown_connections 2024-04-16T15:41:11.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.457+0000 7f336411b640 1 -- 172.21.15.71:0/75540428 wait complete. 2024-04-16T15:41:11.467 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:41:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.571+0000 7fb4dad8f640 1 Processor -- start 2024-04-16T15:41:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.571+0000 7fb4dad8f640 1 -- start start 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4dad8f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4dad8f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb4d40c35f0 0x7fb4d4072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4dad8f640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb4d4072b90 0x7fb4d4155990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4dad8f640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d4152060 con 0x7fb4d4072b90 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4dad8f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d4071a30 con 0x7fb4d40c35f0 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4dad8f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d4071d10 con 0x7fb4d414f780 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d932c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb4d4072b90 0x7fb4d4155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d932c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fb4d4072b90 0x7fb4d4155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57608/0 (socket says 172.21.15.71:57608) 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d932c640 1 -- 172.21.15.71:0/1127577779 learned_addr learned my addr 172.21.15.71:0/1127577779 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:11.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4cbfff640 1 --2- 172.21.15.71:0/1127577779 >> v2:172.21.15.122:3300/0 conn(0x7fb4d40c35f0 0x7fb4d4072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d8b2b640 1 --2- 172.21.15.71:0/1127577779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d932c640 1 -- 172.21.15.71:0/1127577779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 msgr2=0x7fb4d406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d932c640 1 --2- 172.21.15.71:0/1127577779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d932c640 1 -- 172.21.15.71:0/1127577779 >> v2:172.21.15.122:3300/0 conn(0x7fb4d40c35f0 msgr2=0x7fb4d4072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d932c640 1 --2- 172.21.15.71:0/1127577779 >> v2:172.21.15.122:3300/0 conn(0x7fb4d40c35f0 0x7fb4d4072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d932c640 1 -- 172.21.15.71:0/1127577779 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d4071ff0 con 0x7fb4d4072b90 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4cbfff640 1 --2- 172.21.15.71:0/1127577779 >> v2:172.21.15.122:3300/0 conn(0x7fb4d40c35f0 0x7fb4d4072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.572+0000 7fb4d8b2b640 1 --2- 172.21.15.71:0/1127577779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.573+0000 7fb4d932c640 1 --2- 172.21.15.71:0/1127577779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d4072b90 0x7fb4d4155990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4074670 tx=0x7fb4cc02de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69421fa743514f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.573+0000 7fb4cb7fe640 1 -- 172.21.15.71:0/1127577779 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc034070 con 0x7fb4d4072b90 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.573+0000 7fb4cb7fe640 1 -- 172.21.15.71:0/1127577779 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc02ebc0 con 0x7fb4d4072b90 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.573+0000 7fb4cb7fe640 1 -- 172.21.15.71:0/1127577779 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc038400 con 0x7fb4d4072b90 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 -- 172.21.15.71:0/1127577779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d4072b90 msgr2=0x7fb4d4155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/1127577779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d4072b90 0x7fb4d4155990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4074670 tx=0x7fb4cc02de10 comp rx=0 tx=0).stop 2024-04-16T15:41:11.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 -- 172.21.15.71:0/1127577779 shutdown_connections 2024-04-16T15:41:11.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/1127577779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d4072b90 0x7fb4d4155990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/1127577779 >> v2:172.21.15.122:3300/0 conn(0x7fb4d40c35f0 0x7fb4d4072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/1127577779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 -- 172.21.15.71:0/1127577779 >> 172.21.15.71:0/1127577779 conn(0x7fb4d40c3e10 msgr2=0x7fb4d40c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:11.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 -- 172.21.15.71:0/1127577779 shutdown_connections 2024-04-16T15:41:11.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.574+0000 7fb4dad8f640 1 -- 172.21.15.71:0/1127577779 wait complete. 2024-04-16T15:41:11.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.575+0000 7fb4dad8f640 1 Processor -- start 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.575+0000 7fb4dad8f640 1 -- start start 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4dad8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4d4072b90 0x7fb4d40bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4dad8f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d40c35f0 0x7fb4d414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4dad8f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d414d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4dad8f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d406fe70 con 0x7fb4d40c35f0 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4dad8f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d414fb60 con 0x7fb4d4072b90 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4dad8f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d40be340 con 0x7fb4d414f780 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d40c35f0 0x7fb4d414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d40c35f0 0x7fb4d414caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57622/0 (socket says 172.21.15.71:57622) 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 -- 172.21.15.71:0/2337699287 learned_addr learned my addr 172.21.15.71:0/2337699287 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4d932c640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d414d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4d8b2b640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4d4072b90 0x7fb4d40bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 -- 172.21.15.71:0/2337699287 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 msgr2=0x7fb4d414d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d414d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 -- 172.21.15.71:0/2337699287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4d4072b90 msgr2=0x7fb4d40bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4d4072b90 0x7fb4d40bcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 -- 172.21.15.71:0/2337699287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d4154780 con 0x7fb4d40c35f0 2024-04-16T15:41:11.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4cbfff640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d40c35f0 0x7fb4d414caf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400d800 tx=0x7fb4c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4c97fa640 1 -- 172.21.15.71:0/2337699287 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4018070 con 0x7fb4d40c35f0 2024-04-16T15:41:11.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4c97fa640 1 -- 172.21.15.71:0/2337699287 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c4004030 con 0x7fb4d40c35f0 2024-04-16T15:41:11.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4c97fa640 1 -- 172.21.15.71:0/2337699287 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4004400 con 0x7fb4d40c35f0 2024-04-16T15:41:11.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.576+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d4154a60 con 0x7fb4d40c35f0 2024-04-16T15:41:11.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.577+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d40ab690 con 0x7fb4d40c35f0 2024-04-16T15:41:11.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.578+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d4068d20 con 0x7fb4d40c35f0 2024-04-16T15:41:11.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.578+0000 7fb4c97fa640 1 -- 172.21.15.71:0/2337699287 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4c40096a0 con 0x7fb4d40c35f0 2024-04-16T15:41:11.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.578+0000 7fb4c97fa640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb4c006c8f0 0x7fb4c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.579+0000 7fb4c97fa640 1 -- 172.21.15.71:0/2337699287 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fb4c408c120 con 0x7fb4d40c35f0 2024-04-16T15:41:11.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.579+0000 7fb4d8b2b640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb4c006c8f0 0x7fb4c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.579+0000 7fb4d8b2b640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb4c006c8f0 0x7fb4c006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009fd0 tx=0x7fb4d00097f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.583+0000 7fb4c97fa640 1 -- 172.21.15.71:0/2337699287 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4c4059b70 con 0x7fb4d40c35f0 2024-04-16T15:41:11.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.793+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4d406d680 con 0x7fb4d40c35f0 2024-04-16T15:41:11.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.794+0000 7fb4c97fa640 1 -- 172.21.15.71:0/2337699287 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fb4c405de70 con 0x7fb4d40c35f0 2024-04-16T15:41:11.793 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:11.793 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3056194289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3056194289},{"type":"v1","addr":"172.21.15.122:6834","nonce":3056194289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/4245561056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4245561056},{"type":"v1","addr":"172.21.15.71:6835","nonce":4245561056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1205798952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1205798952},{"type":"v1","addr":"172.21.15.71:6837","nonce":1205798952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:40:23.878007+0000","modified":"2024-04-16T15:40:40.120873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/153100987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":153100987},{"type":"v1","addr":"172.21.15.122:6835","nonce":153100987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-04-16T15:41:11.793 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb4c006c8f0 msgr2=0x7fb4c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb4c006c8f0 0x7fb4c006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009fd0 tx=0x7fb4d00097f0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d40c35f0 msgr2=0x7fb4d414caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d40c35f0 0x7fb4d414caf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400d800 tx=0x7fb4c400dcc0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 shutdown_connections 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fb4c006c8f0 0x7fb4c006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb4d414f780 0x7fb4d414d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4d40c35f0 0x7fb4d414caf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 --2- 172.21.15.71:0/2337699287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4d4072b90 0x7fb4d40bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 >> 172.21.15.71:0/2337699287 conn(0x7fb4d40c3e10 msgr2=0x7fb4d414fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:11.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 shutdown_connections 2024-04-16T15:41:11.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.796+0000 7fb4dad8f640 1 -- 172.21.15.71:0/2337699287 wait complete. 2024-04-16T15:41:11.806 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:11.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.910+0000 7fa0d24ba640 1 Processor -- start 2024-04-16T15:41:11.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.910+0000 7fa0d24ba640 1 -- start start 2024-04-16T15:41:11.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d24ba640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa0cc06ff70 0x7fa0cc154490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d24ba640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0cc0703e0 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d24ba640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0cc0a9930 con 0x7fa0cc0c39a0 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d24ba640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fa0cc1549d0 con 0x7fa0cc06ff70 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d0a57640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa0cc06ff70 0x7fa0cc154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0cb7fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fa0cc0c30c0 0x7fa0cc06f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48136/0 (socket says 172.21.15.71:48136) 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d0a57640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa0cc06ff70 0x7fa0cc154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57624/0 (socket says 172.21.15.71:57624) 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0cb7fe640 1 -- 172.21.15.71:0/1580044034 learned_addr learned my addr 172.21.15.71:0/1580044034 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d0a57640 1 -- 172.21.15.71:0/1580044034 >> v1:172.21.15.122:6789/0 conn(0x7fa0cc0c39a0 legacy=0x7fa0cc150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d0a57640 1 -- 172.21.15.71:0/1580044034 >> v1:172.21.15.71:6789/0 conn(0x7fa0cc0c30c0 legacy=0x7fa0cc06f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.911+0000 7fa0d0a57640 1 -- 172.21.15.71:0/1580044034 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc0a9c10 con 0x7fa0cc06ff70 2024-04-16T15:41:11.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.912+0000 7fa0d0a57640 1 --2- 172.21.15.71:0/1580044034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc06ff70 0x7fa0cc154490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009d90 tx=0x7fa0c402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f22a7681bb3495 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0caffd640 1 -- 172.21.15.71:0/1580044034 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4039070 con 0x7fa0cc06ff70 2024-04-16T15:41:11.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0caffd640 1 -- 172.21.15.71:0/1580044034 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c402cbd0 con 0x7fa0cc06ff70 2024-04-16T15:41:11.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0caffd640 1 -- 172.21.15.71:0/1580044034 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4034440 con 0x7fa0cc06ff70 2024-04-16T15:41:11.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1580044034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc06ff70 msgr2=0x7fa0cc154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0d24ba640 1 --2- 172.21.15.71:0/1580044034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc06ff70 0x7fa0cc154490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009d90 tx=0x7fa0c402be20 comp rx=0 tx=0).stop 2024-04-16T15:41:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1580044034 shutdown_connections 2024-04-16T15:41:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0d24ba640 1 --2- 172.21.15.71:0/1580044034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc06ff70 0x7fa0cc154490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1580044034 >> 172.21.15.71:0/1580044034 conn(0x7fa0cc0c4e20 msgr2=0x7fa0cc0c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1580044034 shutdown_connections 2024-04-16T15:41:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.915+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1580044034 wait complete. 2024-04-16T15:41:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.916+0000 7fa0d24ba640 1 Processor -- start 2024-04-16T15:41:11.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0d24ba640 1 -- start start 2024-04-16T15:41:11.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0d24ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa0cc06ff70 0x7fa0cc0c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0d24ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc0c30c0 0x7fa0cc0ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0d24ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc0c39a0 0x7fa0cc0aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0d24ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc0aa4b0 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0d24ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc0aa1d0 con 0x7fa0cc0c39a0 2024-04-16T15:41:11.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0d24ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa0cc151360 con 0x7fa0cc06ff70 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc0c30c0 0x7fa0cc0ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0cb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc0c30c0 0x7fa0cc0ab4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57636/0 (socket says 172.21.15.71:57636) 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.917+0000 7fa0cb7fe640 1 -- 172.21.15.71:0/1893435034 learned_addr learned my addr 172.21.15.71:0/1893435034 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0d0a57640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc0c39a0 0x7fa0cc0aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0cbfff640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa0cc06ff70 0x7fa0cc0c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0cb7fe640 1 -- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa0cc06ff70 msgr2=0x7fa0cc0c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0cb7fe640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa0cc06ff70 0x7fa0cc0c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0cb7fe640 1 -- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc0c39a0 msgr2=0x7fa0cc0aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0cb7fe640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc0c39a0 0x7fa0cc0aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0cb7fe640 1 -- 172.21.15.71:0/1893435034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc0ad2a0 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0cbfff640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa0cc06ff70 0x7fa0cc0c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:11.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0cb7fe640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc0c30c0 0x7fa0cc0ab4d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c4b0 tx=0x7fa0c000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0c8ff9640 1 -- 172.21.15.71:0/1893435034 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0017070 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0c8ff9640 1 -- 172.21.15.71:0/1893435034 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c0005ce0 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0c8ff9640 1 -- 172.21.15.71:0/1893435034 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0007270 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0cc0c2300 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.918+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0cc1ec550 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.919+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0cc068cf0 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.920+0000 7fa0c8ff9640 1 -- 172.21.15.71:0/1893435034 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0c001a070 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.920+0000 7fa0c8ff9640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0b806c920 0x7fa0b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:11.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.920+0000 7fa0c8ff9640 1 -- 172.21.15.71:0/1893435034 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fa0c008b170 con 0x7fa0cc0c30c0 2024-04-16T15:41:11.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.920+0000 7fa0cbfff640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0b806c920 0x7fa0b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:11.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.920+0000 7fa0cbfff640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0b806c920 0x7fa0b806ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0015e0 tx=0x7fa0bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:11.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:11.924+0000 7fa0c8ff9640 1 -- 172.21.15.71:0/1893435034 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0c0058bc0 con 0x7fa0cc0c30c0 2024-04-16T15:41:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.100+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0cc0c20e0 con 0x7fa0cc0c30c0 2024-04-16T15:41:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.101+0000 7fa0c8ff9640 1 -- 172.21.15.71:0/1893435034 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18938 (secure 0 0 0) 0x7fa0c005cec0 con 0x7fa0cc0c30c0 2024-04-16T15:41:12.100 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:12.101 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":26,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:04.856962+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:40:21.794043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T15:40:22.833304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T15:40:26.544818+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:12.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0b806c920 msgr2=0x7fa0b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0b806c920 0x7fa0b806ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0015e0 tx=0x7fa0bc009040 comp rx=0 tx=0).stop 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc0c30c0 msgr2=0x7fa0cc0ab4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc0c30c0 0x7fa0cc0ab4d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c4b0 tx=0x7fa0c000c970 comp rx=0 tx=0).stop 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 shutdown_connections 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0b806c920 0x7fa0b806ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0cc0c39a0 0x7fa0cc0aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0cc0c30c0 0x7fa0cc0ab4d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 --2- 172.21.15.71:0/1893435034 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa0cc06ff70 0x7fa0cc0c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.103+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 >> 172.21.15.71:0/1893435034 conn(0x7fa0cc0c4e20 msgr2=0x7fa0cc14d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.104+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 shutdown_connections 2024-04-16T15:41:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.104+0000 7fa0d24ba640 1 -- 172.21.15.71:0/1893435034 wait complete. 2024-04-16T15:41:12.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-16T15:41:12.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.219+0000 7f979d814640 1 Processor -- start 2024-04-16T15:41:12.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.219+0000 7f979d814640 1 -- start start 2024-04-16T15:41:12.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f979d814640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f97980caf00 0x7f97980ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f979d814640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97980ca910 0x7f97980c0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f979d814640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9798155c00 con 0x7f9798153300 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f979d814640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f97981567a0 con 0x7f97980caf00 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f979d814640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9798155e70 con 0x7f97980ca910 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f97977fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97980ca910 0x7f97980c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f9796ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f9798153300 0x7f9798159af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:39884/0 (socket says 172.21.15.71:39884) 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f9796ffd640 1 -- 172.21.15.71:0/3905179077 learned_addr learned my addr 172.21.15.71:0/3905179077 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f978e7fc640 1 --2- 172.21.15.71:0/3905179077 >> v2:172.21.15.122:3300/0 conn(0x7f97980caf00 0x7f97980ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f97977fe640 1 -- 172.21.15.71:0/3905179077 >> v2:172.21.15.122:3300/0 conn(0x7f97980caf00 msgr2=0x7f97980ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f97977fe640 1 --2- 172.21.15.71:0/3905179077 >> v2:172.21.15.122:3300/0 conn(0x7f97980caf00 0x7f97980ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f97977fe640 1 -- 172.21.15.71:0/3905179077 >> v1:172.21.15.122:6789/0 conn(0x7f9798153300 legacy=0x7f9798159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f97977fe640 1 -- 172.21.15.71:0/3905179077 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9798156150 con 0x7f97980ca910 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f97967fc640 1 -- 172.21.15.71:0/3905179077 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443627323 0 0) 0x7f9798155c00 con 0x7f9798153300 2024-04-16T15:41:12.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.220+0000 7f978e7fc640 1 --2- 172.21.15.71:0/3905179077 >> v2:172.21.15.122:3300/0 conn(0x7f97980caf00 0x7f97980ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:12.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.221+0000 7f97977fe640 1 --2- 172.21.15.71:0/3905179077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97980ca910 0x7f97980c0360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9790009d90 tx=0x7f979002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d51f447c9b1827b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:12.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.221+0000 7f97967fc640 1 -- 172.21.15.71:0/3905179077 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790039070 con 0x7f97980ca910 2024-04-16T15:41:12.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.221+0000 7f97967fc640 1 -- 172.21.15.71:0/3905179077 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979002cbc0 con 0x7f97980ca910 2024-04-16T15:41:12.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.221+0000 7f97967fc640 1 -- 172.21.15.71:0/3905179077 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790034440 con 0x7f97980ca910 2024-04-16T15:41:12.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.222+0000 7f979d814640 1 -- 172.21.15.71:0/3905179077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97980ca910 msgr2=0x7f97980c0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.222+0000 7f979d814640 1 --2- 172.21.15.71:0/3905179077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97980ca910 0x7f97980c0360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9790009d90 tx=0x7f979002be10 comp rx=0 tx=0).stop 2024-04-16T15:41:12.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.222+0000 7f979d814640 1 -- 172.21.15.71:0/3905179077 shutdown_connections 2024-04-16T15:41:12.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.222+0000 7f979d814640 1 --2- 172.21.15.71:0/3905179077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97980ca910 0x7f97980c0360 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.222+0000 7f979d814640 1 --2- 172.21.15.71:0/3905179077 >> v2:172.21.15.122:3300/0 conn(0x7f97980caf00 0x7f97980ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.222+0000 7f979d814640 1 -- 172.21.15.71:0/3905179077 >> 172.21.15.71:0/3905179077 conn(0x7f97980c7f40 msgr2=0x7f97980c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:12.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.222+0000 7f979d814640 1 -- 172.21.15.71:0/3905179077 shutdown_connections 2024-04-16T15:41:12.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.222+0000 7f979d814640 1 -- 172.21.15.71:0/3905179077 wait complete. 2024-04-16T15:41:12.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.223+0000 7f979d814640 1 Processor -- start 2024-04-16T15:41:12.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.223+0000 7f979d814640 1 -- start start 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f979d814640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97980ca910 0x7f97981e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f979d814640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980caf00 0x7f9798148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f979d814640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9798153300 0x7f97981492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f979d814640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f97980c0a20 con 0x7f97980ca910 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f979d814640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97980c08a0 con 0x7f97980caf00 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f979d814640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f97980c0ba0 con 0x7f9798153300 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f9796ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97980ca910 0x7f97981e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f9796ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97980ca910 0x7f97981e2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57646/0 (socket says 172.21.15.71:57646) 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f9796ffd640 1 -- 172.21.15.71:0/894749541 learned_addr learned my addr 172.21.15.71:0/894749541 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:12.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f978e7fc640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980caf00 0x7f9798148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f9796ffd640 1 -- 172.21.15.71:0/894749541 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9798153300 msgr2=0x7f97981492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f9796ffd640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9798153300 0x7f97981492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f9796ffd640 1 -- 172.21.15.71:0/894749541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980caf00 msgr2=0x7f9798148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f9796ffd640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980caf00 0x7f9798148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.224+0000 7f9796ffd640 1 -- 172.21.15.71:0/894749541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9798155610 con 0x7f97980ca910 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.225+0000 7f9796ffd640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97980ca910 0x7f97981e2c10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f978800c430 tx=0x7f978800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.225+0000 7f9794ff9640 1 -- 172.21.15.71:0/894749541 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97880074e0 con 0x7f97980ca910 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.225+0000 7f9794ff9640 1 -- 172.21.15.71:0/894749541 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97880047e0 con 0x7f97980ca910 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.225+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9798149b20 con 0x7f97980ca910 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.225+0000 7f9794ff9640 1 -- 172.21.15.71:0/894749541 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788004bd0 con 0x7f97980ca910 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.225+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97981e3150 con 0x7f97980ca910 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.226+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979814c030 con 0x7f97980ca910 2024-04-16T15:41:12.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.226+0000 7f9794ff9640 1 -- 172.21.15.71:0/894749541 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9788011070 con 0x7f97980ca910 2024-04-16T15:41:12.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.227+0000 7f9794ff9640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f977806c8f0 0x7f977806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.227+0000 7f9794ff9640 1 -- 172.21.15.71:0/894749541 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f978808bad0 con 0x7f97980ca910 2024-04-16T15:41:12.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.227+0000 7f978e7fc640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f977806c8f0 0x7f977806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.227+0000 7f978e7fc640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f977806c8f0 0x7f977806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9780009560 tx=0x7f9780008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:12.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.231+0000 7f9794ff9640 1 -- 172.21.15.71:0/894749541 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f978800e280 con 0x7f97980ca910 2024-04-16T15:41:12.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.443+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f97980c8ce0 con 0x7f97980ca910 2024-04-16T15:41:12.442 INFO:tasks.ceph.mon.a.smithi071.stderr:2024-04-16T15:41:12.443+0000 7f2c5ebdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T15:41:12.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.445+0000 7f9794ff9640 1 -- 172.21.15.71:0/894749541 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f9788059b70 con 0x7f97980ca910 2024-04-16T15:41:12.443 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f977806c8f0 msgr2=0x7f977806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f977806c8f0 0x7f977806edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9780009560 tx=0x7f9780008040 comp rx=0 tx=0).stop 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97980ca910 msgr2=0x7f97981e2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97980ca910 0x7f97981e2c10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f978800c430 tx=0x7f978800c8f0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 shutdown_connections 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f977806c8f0 0x7f977806edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9798153300 0x7f97981492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980caf00 0x7f9798148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 --2- 172.21.15.71:0/894749541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97980ca910 0x7f97981e2c10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 >> 172.21.15.71:0/894749541 conn(0x7f97980c7f40 msgr2=0x7f97980c0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:12.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 shutdown_connections 2024-04-16T15:41:12.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.447+0000 7f979d814640 1 -- 172.21.15.71:0/894749541 wait complete. 2024-04-16T15:41:12.449 INFO:tasks.ceph.mds.b.smithi122.stderr: -3639> 2024-04-16T15:40:28.345+0000 7fa0d9c36640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-16T15:41:12.458 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-16T15:41:12.481 INFO:tasks.ceph.mds.b.smithi122.stdout:starting mds.b at 2024-04-16T15:41:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.565+0000 7f54a85f4640 1 Processor -- start 2024-04-16T15:41:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.566+0000 7f54a85f4640 1 -- start start 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.566+0000 7f54a85f4640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f54a0153430 0x7f54a01571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.566+0000 7f54a85f4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 0x7f54a0160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.566+0000 7f54a85f4640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a0156c70 con 0x7f54a0157720 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.566+0000 7f54a85f4640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f54a00c67f0 con 0x7f54a0153430 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.566+0000 7f54a85f4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f54a00c5ec0 con 0x7f54a0156770 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a6390640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f54a0153430 0x7f54a01571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a6390640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f54a0153430 0x7f54a01571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57658/0 (socket says 172.21.15.71:57658) 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a5b8f640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f54a0157720 0x7f54a00ca3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48152/0 (socket says 172.21.15.71:48152) 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a6390640 1 -- 172.21.15.71:0/1187883111 learned_addr learned my addr 172.21.15.71:0/1187883111 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a6b91640 1 --2- 172.21.15.71:0/1187883111 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 0x7f54a0160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a6390640 1 -- 172.21.15.71:0/1187883111 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 msgr2=0x7f54a0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a6390640 1 --2- 172.21.15.71:0/1187883111 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 0x7f54a0160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a6390640 1 -- 172.21.15.71:0/1187883111 >> v1:172.21.15.71:6789/0 conn(0x7f54a0157720 legacy=0x7f54a00ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a6390640 1 -- 172.21.15.71:0/1187883111 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a00c61a0 con 0x7f54a0153430 2024-04-16T15:41:12.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.567+0000 7f54a538e640 1 -- 172.21.15.71:0/1187883111 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 634162707 0 0) 0x7f54a0156c70 con 0x7f54a0157720 2024-04-16T15:41:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a6390640 1 --2- 172.21.15.71:0/1187883111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0153430 0x7f54a01571e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5490009e50 tx=0x7f549002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9840d8f0b1c47062 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a538e640 1 -- 172.21.15.71:0/1187883111 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549002ca70 con 0x7f54a0153430 2024-04-16T15:41:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a538e640 1 -- 172.21.15.71:0/1187883111 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549002cc10 con 0x7f54a0153430 2024-04-16T15:41:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a538e640 1 -- 172.21.15.71:0/1187883111 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54900343f0 con 0x7f54a0153430 2024-04-16T15:41:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a85f4640 1 -- 172.21.15.71:0/1187883111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0153430 msgr2=0x7f54a01571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a85f4640 1 --2- 172.21.15.71:0/1187883111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0153430 0x7f54a01571e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5490009e50 tx=0x7f549002bed0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a85f4640 1 -- 172.21.15.71:0/1187883111 shutdown_connections 2024-04-16T15:41:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a85f4640 1 --2- 172.21.15.71:0/1187883111 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 0x7f54a0160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a85f4640 1 --2- 172.21.15.71:0/1187883111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0153430 0x7f54a01571e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a85f4640 1 -- 172.21.15.71:0/1187883111 >> 172.21.15.71:0/1187883111 conn(0x7f54a00c7f20 msgr2=0x7f54a00c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a85f4640 1 -- 172.21.15.71:0/1187883111 shutdown_connections 2024-04-16T15:41:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.568+0000 7f54a85f4640 1 -- 172.21.15.71:0/1187883111 wait complete. 2024-04-16T15:41:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.569+0000 7f54a85f4640 1 Processor -- start 2024-04-16T15:41:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a85f4640 1 -- start start 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a85f4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54a0153430 0x7f54a014bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a85f4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 0x7f54a00c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a85f4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0157720 0x7f54a00c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a85f4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f54a00cac40 con 0x7f54a0157720 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a85f4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f54a00caac0 con 0x7f54a0153430 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a85f4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f54a0153810 con 0x7f54a0156770 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a6b91640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0157720 0x7f54a00c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a6b91640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0157720 0x7f54a00c2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57670/0 (socket says 172.21.15.71:57670) 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.570+0000 7f54a6b91640 1 -- 172.21.15.71:0/608943667 learned_addr learned my addr 172.21.15.71:0/608943667 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a5b8f640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 0x7f54a00c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a6390640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54a0153430 0x7f54a014bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a6b91640 1 -- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 msgr2=0x7f54a00c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a6b91640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 0x7f54a00c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a6b91640 1 -- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54a0153430 msgr2=0x7f54a014bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a6b91640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54a0153430 0x7f54a014bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a6b91640 1 -- 172.21.15.71:0/608943667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a0154660 con 0x7f54a0157720 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a6390640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54a0153430 0x7f54a014bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a6b91640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0157720 0x7f54a00c2ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f549800d5d0 tx=0x7f549800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f549effd640 1 -- 172.21.15.71:0/608943667 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54980125f0 con 0x7f54a0157720 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f549effd640 1 -- 172.21.15.71:0/608943667 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5498019070 con 0x7f54a0157720 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a0154940 con 0x7f54a0157720 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f549effd640 1 -- 172.21.15.71:0/608943667 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54980129d0 con 0x7f54a0157720 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.571+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a014c730 con 0x7f54a0157720 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.572+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a0073f50 con 0x7f54a0157720 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.572+0000 7f549effd640 1 -- 172.21.15.71:0/608943667 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5498012b70 con 0x7f54a0157720 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.573+0000 7f549effd640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f548c06c8f0 0x7f548c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.573+0000 7f54a6390640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f548c06c8f0 0x7f548c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:12.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.573+0000 7f549effd640 1 -- 172.21.15.71:0/608943667 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6850+0+0 (secure 0 0 0) 0x7f549808c390 con 0x7f54a0157720 2024-04-16T15:41:12.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.573+0000 7f54a6390640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f548c06c8f0 0x7f548c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5490035000 tx=0x7f5490030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:12.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.578+0000 7f549effd640 1 -- 172.21.15.71:0/608943667 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f549805a3d0 con 0x7f54a0157720 2024-04-16T15:41:12.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:12.789+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f54a00aac60 con 0x7f54a0157720 2024-04-16T15:41:13.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.446+0000 7f549effd640 1 -- 172.21.15.71:0/608943667 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f54a00aac60 con 0x7f54a0157720 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.448+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f548c06c8f0 msgr2=0x7f548c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.448+0000 7f54a85f4640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f548c06c8f0 0x7f548c06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5490035000 tx=0x7f5490030000 comp rx=0 tx=0).stop 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.448+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0157720 msgr2=0x7f54a00c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.448+0000 7f54a85f4640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0157720 0x7f54a00c2ab0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f549800d5d0 tx=0x7f549800da90 comp rx=0 tx=0).stop 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.449+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 shutdown_connections 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.449+0000 7f54a85f4640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f548c06c8f0 0x7f548c06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.449+0000 7f54a85f4640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a0157720 0x7f54a00c2ab0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.449+0000 7f54a85f4640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54a0156770 0x7f54a00c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.449+0000 7f54a85f4640 1 --2- 172.21.15.71:0/608943667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54a0153430 0x7f54a014bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.449+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 >> 172.21.15.71:0/608943667 conn(0x7f54a00c7f20 msgr2=0x7f54a00c1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.449+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 shutdown_connections 2024-04-16T15:41:13.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.449+0000 7f54a85f4640 1 -- 172.21.15.71:0/608943667 wait complete. 2024-04-16T15:41:13.460 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-16T15:41:13.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.570+0000 7fcfcbfa8640 1 Processor -- start 2024-04-16T15:41:13.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.570+0000 7fcfcbfa8640 1 -- start start 2024-04-16T15:41:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfcbfa8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcfc414c910 0x7fcfc40c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfcbfa8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcfc415b1c0 0x7fcfc415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:13.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfcbfa8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc4152850 con 0x7fcfc4072840 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfcbfa8640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc414ccf0 con 0x7fcfc415b1c0 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfcbfa8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc4152b10 con 0x7fcfc414c910 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfca545640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcfc415b1c0 0x7fcfc415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfc9543640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fcfc4072840 0x7fcfc4073070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48160/0 (socket says 172.21.15.71:48160) 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfca545640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcfc415b1c0 0x7fcfc415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57684/0 (socket says 172.21.15.71:57684) 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfc9543640 1 -- 172.21.15.71:0/1863604541 learned_addr learned my addr 172.21.15.71:0/1863604541 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfca545640 1 -- 172.21.15.71:0/1863604541 >> v2:172.21.15.122:3300/0 conn(0x7fcfc414c910 msgr2=0x7fcfc40c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfca545640 1 --2- 172.21.15.71:0/1863604541 >> v2:172.21.15.122:3300/0 conn(0x7fcfc414c910 0x7fcfc40c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfca545640 1 -- 172.21.15.71:0/1863604541 >> v1:172.21.15.71:6789/0 conn(0x7fcfc4072840 legacy=0x7fcfc4073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:13.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfca545640 1 -- 172.21.15.71:0/1863604541 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc4071d60 con 0x7fcfc415b1c0 2024-04-16T15:41:13.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.571+0000 7fcfc8d42640 1 -- 172.21.15.71:0/1863604541 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 439908445 0 0) 0x7fcfc4152850 con 0x7fcfc4072840 2024-04-16T15:41:13.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.572+0000 7fcfca545640 1 --2- 172.21.15.71:0/1863604541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc415d5b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4074160 tx=0x7fcfbc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1151990fc32e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:13.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.572+0000 7fcfc8d42640 1 -- 172.21.15.71:0/1863604541 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc035020 con 0x7fcfc415b1c0 2024-04-16T15:41:13.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.572+0000 7fcfc8d42640 1 -- 172.21.15.71:0/1863604541 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfbc031830 con 0x7fcfc415b1c0 2024-04-16T15:41:13.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.572+0000 7fcfc8d42640 1 -- 172.21.15.71:0/1863604541 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc031c00 con 0x7fcfc415b1c0 2024-04-16T15:41:13.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.573+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/1863604541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 msgr2=0x7fcfc415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.573+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/1863604541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc415d5b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4074160 tx=0x7fcfbc02b8d0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.573+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/1863604541 shutdown_connections 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.573+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/1863604541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc415d5b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.573+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/1863604541 >> v2:172.21.15.122:3300/0 conn(0x7fcfc414c910 0x7fcfc40c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.573+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/1863604541 >> 172.21.15.71:0/1863604541 conn(0x7fcfc40c3e80 msgr2=0x7fcfc40c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.573+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/1863604541 shutdown_connections 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.573+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/1863604541 wait complete. 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.574+0000 7fcfcbfa8640 1 Processor -- start 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.574+0000 7fcfcbfa8640 1 -- start start 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.574+0000 7fcfcbfa8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfc4072840 0x7fcfc41eced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.574+0000 7fcfcbfa8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfc414c910 0x7fcfc40ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.574+0000 7fcfcbfa8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:13.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.574+0000 7fcfcbfa8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4150400 con 0x7fcfc415b1c0 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.574+0000 7fcfcbfa8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4072320 con 0x7fcfc414c910 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.574+0000 7fcfcbfa8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc40c19c0 con 0x7fcfc4072840 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc406c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57690/0 (socket says 172.21.15.71:57690) 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 -- 172.21.15.71:0/2962702899 learned_addr learned my addr 172.21.15.71:0/2962702899 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 -- 172.21.15.71:0/2962702899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfc4072840 msgr2=0x7fcfc41eced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfc4072840 0x7fcfc41eced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 -- 172.21.15.71:0/2962702899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfc414c910 msgr2=0x7fcfc40ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfc9543640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfc414c910 0x7fcfc40ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfc414c910 0x7fcfc40ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 -- 172.21.15.71:0/2962702899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc40c64b0 con 0x7fcfc415b1c0 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfc9543640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfc414c910 0x7fcfc40ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:13.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfca545640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc406c1c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009890 tx=0x7fcfbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:13.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfba7fc640 1 -- 172.21.15.71:0/2962702899 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc041070 con 0x7fcfc415b1c0 2024-04-16T15:41:13.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfba7fc640 1 -- 172.21.15.71:0/2962702899 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfbc03a950 con 0x7fcfc415b1c0 2024-04-16T15:41:13.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfba7fc640 1 -- 172.21.15.71:0/2962702899 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc035040 con 0x7fcfc415b1c0 2024-04-16T15:41:13.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc40c6790 con 0x7fcfc415b1c0 2024-04-16T15:41:13.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.575+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc40c7030 con 0x7fcfc415b1c0 2024-04-16T15:41:13.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.576+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc40c6b80 con 0x7fcfc415b1c0 2024-04-16T15:41:13.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.577+0000 7fcfba7fc640 1 -- 172.21.15.71:0/2962702899 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfbc038020 con 0x7fcfc415b1c0 2024-04-16T15:41:13.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.577+0000 7fcfba7fc640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfa806c920 0x7fcfa806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:13.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.577+0000 7fcfba7fc640 1 -- 172.21.15.71:0/2962702899 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6850+0+0 (secure 0 0 0) 0x7fcfbc0ada60 con 0x7fcfc415b1c0 2024-04-16T15:41:13.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.577+0000 7fcfc9d44640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfa806c920 0x7fcfa806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:13.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.578+0000 7fcfc9d44640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfa806c920 0x7fcfa806ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000ab90 tx=0x7fcfc0009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:13.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.582+0000 7fcfba7fc640 1 -- 172.21.15.71:0/2962702899 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfbc07ba10 con 0x7fcfc415b1c0 2024-04-16T15:41:13.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:13.758+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcfc4068be0 con 0x7fcfc415b1c0 2024-04-16T15:41:14.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.451+0000 7fcfba7fc640 1 -- 172.21.15.71:0/2962702899 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7fcfbc07b310 con 0x7fcfc415b1c0 2024-04-16T15:41:14.450 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-04-16T15:41:14.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfa806c920 msgr2=0x7fcfa806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:14.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfa806c920 0x7fcfa806ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000ab90 tx=0x7fcfc0009250 comp rx=0 tx=0).stop 2024-04-16T15:41:14.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 msgr2=0x7fcfc406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc406c1c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009890 tx=0x7fcfbc0046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 shutdown_connections 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcfa806c920 0x7fcfa806ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc415b1c0 0x7fcfc406c1c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfc414c910 0x7fcfc40ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 --2- 172.21.15.71:0/2962702899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfc4072840 0x7fcfc41eced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.453+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 >> 172.21.15.71:0/2962702899 conn(0x7fcfc40c3e80 msgr2=0x7fcfc40c35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.454+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 shutdown_connections 2024-04-16T15:41:14.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.454+0000 7fcfcbfa8640 1 -- 172.21.15.71:0/2962702899 wait complete. 2024-04-16T15:41:14.467 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-16T15:41:14.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.578+0000 7fe11b437640 1 Processor -- start 2024-04-16T15:41:14.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.578+0000 7fe11b437640 1 -- start start 2024-04-16T15:41:14.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe11b437640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe11406fff0 0x7fe1140c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:14.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe11b437640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1140c5a30 con 0x7fe114154830 2024-04-16T15:41:14.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe11b437640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe114153d50 con 0x7fe1140c8de0 2024-04-16T15:41:14.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe11b437640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe114154030 con 0x7fe11406fff0 2024-04-16T15:41:14.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe1189d2640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fe114154830 0x7fe1140bfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48174/0 (socket says 172.21.15.71:48174) 2024-04-16T15:41:14.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe1189d2640 1 -- 172.21.15.71:0/3351200148 learned_addr learned my addr 172.21.15.71:0/3351200148 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:14.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe1199d4640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe11406fff0 0x7fe1140c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 824027387 0 0) 0x7fe1140c5a30 con 0x7fe114154830 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe100003520 con 0x7fe114154830 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160860157 0 0) 0x7fe114153d50 con 0x7fe1140c8de0 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1140c5a30 con 0x7fe1140c8de0 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3908380722 0 0) 0x7fe100003520 con 0x7fe114154830 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe11406fff0 msgr2=0x7fe1140c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe11406fff0 0x7fe1140c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 >> v1:172.21.15.122:6789/0 conn(0x7fe1140c8de0 legacy=0x7fe11406ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe114154310 con 0x7fe114154830 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fe104001670 con 0x7fe114154830 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 >> v1:172.21.15.71:6789/0 conn(0x7fe114154830 legacy=0x7fe1140bfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe100003b70 0x7fe100003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe100004800 0x7fe100004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe100005210 0x7fe10000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe114154030 con 0x7fe100004800 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe10000c0f0 con 0x7fe100003b70 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe10000c2f0 con 0x7fe100005210 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe1199d4640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe11406fff0 0x7fe1140c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe1189d2640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe100004800 0x7fe100004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.579+0000 7fe1191d3640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe100003b70 0x7fe100003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.580+0000 7fe1199d4640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe100005210 0x7fe10000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.580+0000 7fe1189d2640 1 -- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe100005210 msgr2=0x7fe10000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.580+0000 7fe1189d2640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe100005210 0x7fe10000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.580+0000 7fe1189d2640 1 -- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe100003b70 msgr2=0x7fe100003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.580+0000 7fe1189d2640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe100003b70 0x7fe100003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.580+0000 7fe1189d2640 1 -- 172.21.15.71:0/3351200148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe100003520 con 0x7fe100004800 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.580+0000 7fe1189d2640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe100004800 0x7fe100004c80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c160 tx=0x7fe100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79feac7595dfbfd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104007ca0 con 0x7fe100004800 2024-04-16T15:41:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe104030070 con 0x7fe100004800 2024-04-16T15:41:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe10bfff640 1 -- 172.21.15.71:0/3351200148 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104005d30 con 0x7fe100004800 2024-04-16T15:41:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe11b437640 1 -- 172.21.15.71:0/3351200148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe100004800 msgr2=0x7fe100004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe11b437640 1 --2- 172.21.15.71:0/3351200148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe100004800 0x7fe100004c80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c160 tx=0x7fe100005830 comp rx=0 tx=0).stop 2024-04-16T15:41:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe1199d4640 1 -- 172.21.15.71:0/3351200148 reap_dead start 2024-04-16T15:41:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe11b437640 1 -- 172.21.15.71:0/3351200148 shutdown_connections 2024-04-16T15:41:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe11b437640 1 -- 172.21.15.71:0/3351200148 >> 172.21.15.71:0/3351200148 conn(0x7fe1140c5e20 msgr2=0x7fe1140c6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe11b437640 1 -- 172.21.15.71:0/3351200148 shutdown_connections 2024-04-16T15:41:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.581+0000 7fe11b437640 1 -- 172.21.15.71:0/3351200148 wait complete. 2024-04-16T15:41:14.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.582+0000 7fe11b437640 1 Processor -- start 2024-04-16T15:41:14.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.582+0000 7fe11b437640 1 -- start start 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe11b437640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe100004800 0x7fe1140aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe11b437640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11406dd00 0x7fe11414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe11b437640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe11414a8e0 0x7fe1141eae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe11b437640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe1140c0360 con 0x7fe11406dd00 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe11b437640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe11406f360 con 0x7fe11414a8e0 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe11b437640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe1140c04e0 con 0x7fe100004800 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe1189d2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11406dd00 0x7fe11414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe1189d2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11406dd00 0x7fe11414a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57712/0 (socket says 172.21.15.71:57712) 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe1189d2640 1 -- 172.21.15.71:0/505739286 learned_addr learned my addr 172.21.15.71:0/505739286 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe1189d2640 1 -- 172.21.15.71:0/505739286 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe100004800 msgr2=0x7fe1140aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe1189d2640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe100004800 0x7fe1140aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe1189d2640 1 -- 172.21.15.71:0/505739286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe11414a8e0 msgr2=0x7fe1141eae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe1189d2640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe11414a8e0 0x7fe1141eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:14.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.583+0000 7fe1189d2640 1 -- 172.21.15.71:0/505739286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11406e5a0 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.584+0000 7fe1189d2640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11406dd00 0x7fe11414a3a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c160 tx=0x7fe1140c4a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.584+0000 7fe109ffb640 1 -- 172.21.15.71:0/505739286 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104001c00 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.584+0000 7fe109ffb640 1 -- 172.21.15.71:0/505739286 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1040049e0 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.584+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1140c30d0 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.584+0000 7fe109ffb640 1 -- 172.21.15.71:0/505739286 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104004d90 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.584+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1140c3690 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.585+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe114068be0 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.585+0000 7fe109ffb640 1 -- 172.21.15.71:0/505739286 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe104047020 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.586+0000 7fe109ffb640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0f806c8f0 0x7fe0f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.586+0000 7fe109ffb640 1 -- 172.21.15.71:0/505739286 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6412+0+0 (secure 0 0 0) 0x7fe1040acdb0 con 0x7fe11406dd00 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.586+0000 7fe1191d3640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0f806c8f0 0x7fe0f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:14.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.586+0000 7fe1191d3640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0f806c8f0 0x7fe0f806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe110001760 tx=0x7fe110009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:14.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.590+0000 7fe109ffb640 1 -- 172.21.15.71:0/505739286 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe10407af20 con 0x7fe11406dd00 2024-04-16T15:41:14.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:14.764+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe1140af670 con 0x7fe11406dd00 2024-04-16T15:41:15.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.452+0000 7fe109ffb640 1 -- 172.21.15.71:0/505739286 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7fe10407a820 con 0x7fe11406dd00 2024-04-16T15:41:15.451 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-04-16T15:41:15.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.456+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0f806c8f0 msgr2=0x7fe0f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:15.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.456+0000 7fe11b437640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0f806c8f0 0x7fe0f806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe110001760 tx=0x7fe110009040 comp rx=0 tx=0).stop 2024-04-16T15:41:15.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.456+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11406dd00 msgr2=0x7fe11414a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:15.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.456+0000 7fe11b437640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11406dd00 0x7fe11414a3a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c160 tx=0x7fe1140c4a20 comp rx=0 tx=0).stop 2024-04-16T15:41:15.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.457+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 shutdown_connections 2024-04-16T15:41:15.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.457+0000 7fe11b437640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe0f806c8f0 0x7fe0f806edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.457+0000 7fe11b437640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe11414a8e0 0x7fe1141eae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.457+0000 7fe11b437640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11406dd00 0x7fe11414a3a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.457+0000 7fe11b437640 1 --2- 172.21.15.71:0/505739286 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe100004800 0x7fe1140aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.457+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 >> 172.21.15.71:0/505739286 conn(0x7fe1140c5e20 msgr2=0x7fe114071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:15.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.457+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 shutdown_connections 2024-04-16T15:41:15.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.458+0000 7fe11b437640 1 -- 172.21.15.71:0/505739286 wait complete. 2024-04-16T15:41:15.476 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-16T15:41:15.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.588+0000 7ff6de504640 1 Processor -- start 2024-04-16T15:41:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.588+0000 7ff6de504640 1 -- start start 2024-04-16T15:41:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.588+0000 7ff6de504640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff6d8073670 0x7ff6d8073a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6de504640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7ff6d80c3880 0x7ff6d8153950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6de504640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3080 0x7ff6d8153e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6de504640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d80c4060 con 0x7ff6d80c3880 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6de504640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d80adfb0 con 0x7ff6d8073670 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6de504640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d80c3d80 con 0x7ff6d80c3080 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d77fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7ff6d80c3880 0x7ff6d8153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d77fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7ff6d80c3880 0x7ff6d8153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57728/0 (socket says 172.21.15.71:57728) 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d77fe640 1 -- 172.21.15.71:0/2014591533 learned_addr learned my addr 172.21.15.71:0/2014591533 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6dcaa1640 1 --2- 172.21.15.71:0/2014591533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3080 0x7ff6d8153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d7fff640 1 --2- 172.21.15.71:0/2014591533 >> v2:172.21.15.122:3300/0 conn(0x7ff6d8073670 0x7ff6d8073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d77fe640 1 -- 172.21.15.71:0/2014591533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3080 msgr2=0x7ff6d8153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d77fe640 1 --2- 172.21.15.71:0/2014591533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3080 0x7ff6d8153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d77fe640 1 -- 172.21.15.71:0/2014591533 >> v2:172.21.15.122:3300/0 conn(0x7ff6d8073670 msgr2=0x7ff6d8073a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d77fe640 1 --2- 172.21.15.71:0/2014591533 >> v2:172.21.15.122:3300/0 conn(0x7ff6d8073670 0x7ff6d8073a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d77fe640 1 -- 172.21.15.71:0/2014591533 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d80ad960 con 0x7ff6d80c3880 2024-04-16T15:41:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6dcaa1640 1 --2- 172.21.15.71:0/2014591533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3080 0x7ff6d8153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.589+0000 7ff6d7fff640 1 --2- 172.21.15.71:0/2014591533 >> v2:172.21.15.122:3300/0 conn(0x7ff6d8073670 0x7ff6d8073a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6d77fe640 1 --2- 172.21.15.71:0/2014591533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3880 0x7ff6d8153950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff6d806e820 tx=0x7ff6cc02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6ab6a7fe555c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6d6ffd640 1 -- 172.21.15.71:0/2014591533 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc038070 con 0x7ff6d80c3880 2024-04-16T15:41:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6d6ffd640 1 -- 172.21.15.71:0/2014591533 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6cc02f050 con 0x7ff6d80c3880 2024-04-16T15:41:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6d6ffd640 1 -- 172.21.15.71:0/2014591533 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc031970 con 0x7ff6d80c3880 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6de504640 1 -- 172.21.15.71:0/2014591533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3880 msgr2=0x7ff6d8153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6de504640 1 --2- 172.21.15.71:0/2014591533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3880 0x7ff6d8153950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff6d806e820 tx=0x7ff6cc02b7f0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6de504640 1 -- 172.21.15.71:0/2014591533 shutdown_connections 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6de504640 1 --2- 172.21.15.71:0/2014591533 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3080 0x7ff6d8153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6de504640 1 --2- 172.21.15.71:0/2014591533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3880 0x7ff6d8153950 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6de504640 1 --2- 172.21.15.71:0/2014591533 >> v2:172.21.15.122:3300/0 conn(0x7ff6d8073670 0x7ff6d8073a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.590+0000 7ff6de504640 1 -- 172.21.15.71:0/2014591533 >> 172.21.15.71:0/2014591533 conn(0x7ff6d80aa220 msgr2=0x7ff6d80aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.591+0000 7ff6de504640 1 -- 172.21.15.71:0/2014591533 shutdown_connections 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.591+0000 7ff6de504640 1 -- 172.21.15.71:0/2014591533 wait complete. 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.592+0000 7ff6de504640 1 Processor -- start 2024-04-16T15:41:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.592+0000 7ff6de504640 1 -- start start 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.592+0000 7ff6de504640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6d8073670 0x7ff6d814cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.592+0000 7ff6de504640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3080 0x7ff6d81e03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.592+0000 7ff6de504640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3880 0x7ff6d81ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.592+0000 7ff6de504640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81511b0 con 0x7ff6d80c3080 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.592+0000 7ff6de504640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d80c2a10 con 0x7ff6d8073670 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.592+0000 7ff6de504640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d8151330 con 0x7ff6d80c3880 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3080 0x7ff6d81e03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3080 0x7ff6d81e03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57736/0 (socket says 172.21.15.71:57736) 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 -- 172.21.15.71:0/3877016319 learned_addr learned my addr 172.21.15.71:0/3877016319 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d7fff640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6d8073670 0x7ff6d814cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6dcaa1640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3880 0x7ff6d81ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 -- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3880 msgr2=0x7ff6d81ecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3880 0x7ff6d81ecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 -- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6d8073670 msgr2=0x7ff6d814cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6d8073670 0x7ff6d814cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 -- 172.21.15.71:0/3877016319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d8152220 con 0x7ff6d80c3080 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6dcaa1640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3880 0x7ff6d81ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d77fe640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3080 0x7ff6d81e03c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc000c00 tx=0x7ff6cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d4ff9640 1 -- 172.21.15.71:0/3877016319 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc042070 con 0x7ff6d80c3080 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d4ff9640 1 -- 172.21.15.71:0/3877016319 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6cc03d890 con 0x7ff6d80c3080 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d8152500 con 0x7ff6d80c3080 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6d4ff9640 1 -- 172.21.15.71:0/3877016319 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc038040 con 0x7ff6d80c3080 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.593+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d8149100 con 0x7ff6d80c3080 2024-04-16T15:41:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.594+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d80c3660 con 0x7ff6d80c3080 2024-04-16T15:41:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.595+0000 7ff6d4ff9640 1 -- 172.21.15.71:0/3877016319 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6cc045070 con 0x7ff6d80c3080 2024-04-16T15:41:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.595+0000 7ff6d4ff9640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6c406c8f0 0x7ff6c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.595+0000 7ff6d4ff9640 1 -- 172.21.15.71:0/3877016319 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7ff6cc0ad750 con 0x7ff6d80c3080 2024-04-16T15:41:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.595+0000 7ff6d7fff640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6c406c8f0 0x7ff6c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.596+0000 7ff6d7fff640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6c406c8f0 0x7ff6c406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff6d81e77e0 tx=0x7ff6c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:15.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.600+0000 7ff6d4ff9640 1 -- 172.21.15.71:0/3877016319 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6cc07b5d0 con 0x7ff6d80c3080 2024-04-16T15:41:15.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:15.778+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6d80aadd0 con 0x7ff6d80c3080 2024-04-16T15:41:16.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.467+0000 7ff6d4ff9640 1 -- 172.21.15.71:0/3877016319 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7ff6cc07f8d0 con 0x7ff6d80c3080 2024-04-16T15:41:16.466 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data_ec' removed 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.470+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6c406c8f0 msgr2=0x7ff6c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6c406c8f0 0x7ff6c406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff6d81e77e0 tx=0x7ff6c8008040 comp rx=0 tx=0).stop 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3080 msgr2=0x7ff6d81e03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3080 0x7ff6d81e03c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc000c00 tx=0x7ff6cc0046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 shutdown_connections 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6c406c8f0 0x7ff6c406edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff6d80c3880 0x7ff6d81ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6d80c3080 0x7ff6d81e03c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 --2- 172.21.15.71:0/3877016319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6d8073670 0x7ff6d814cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 >> 172.21.15.71:0/3877016319 conn(0x7ff6d80aa220 msgr2=0x7ff6d8150300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:16.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 shutdown_connections 2024-04-16T15:41:16.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.471+0000 7ff6de504640 1 -- 172.21.15.71:0/3877016319 wait complete. 2024-04-16T15:41:16.485 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-16T15:41:16.485 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-16T15:41:16.485 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-16T15:41:16.485 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:16.486 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-16T15:41:16.486 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-16T15:41:16.486 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-16T15:41:16.486 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:16.486 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-16T15:41:16.486 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-16T15:41:16.486 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-16T15:41:16.486 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:16.487 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-16T15:41:16.487 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-16T15:41:16.487 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-16T15:41:16.487 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:16.496 INFO:tasks.ceph.mds.a:Stopped 2024-04-16T15:41:16.497 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-16T15:41:16.497 INFO:tasks.ceph.mds.c:Stopped 2024-04-16T15:41:16.497 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-16T15:41:16.498 INFO:tasks.ceph.mds.b:Stopped 2024-04-16T15:41:16.499 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-16T15:41:16.499 INFO:tasks.ceph.mds.d:Stopped 2024-04-16T15:41:16.499 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-16T15:41:16.539 INFO:tasks.ceph.mds.a:Started 2024-04-16T15:41:16.540 DEBUG:teuthology.parallel:result is None 2024-04-16T15:41:16.541 INFO:tasks.ceph.mds.c:Started 2024-04-16T15:41:16.541 DEBUG:teuthology.parallel:result is None 2024-04-16T15:41:16.542 INFO:tasks.ceph.mds.b:Started 2024-04-16T15:41:16.542 DEBUG:teuthology.parallel:result is None 2024-04-16T15:41:16.543 INFO:tasks.ceph.mds.d:Started 2024-04-16T15:41:16.543 DEBUG:teuthology.parallel:result is None 2024-04-16T15:41:16.543 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-16T15:41:16.630 INFO:tasks.ceph.mds.d.smithi122.stdout:starting mds.d at 2024-04-16T15:41:16.630 INFO:tasks.ceph.mds.b.smithi122.stdout:starting mds.b at 2024-04-16T15:41:16.631 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2024-04-16T15:41:16.631 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2024-04-16T15:41:16.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.665+0000 7fccf0f0e640 1 Processor -- start 2024-04-16T15:41:16.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.665+0000 7fccf0f0e640 1 -- start start 2024-04-16T15:41:16.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.666+0000 7fccf0f0e640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fccec153300 0x7fccec159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:16.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.666+0000 7fccf0f0e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fccec0caf00 0x7fccec0ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:16.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.666+0000 7fccf0f0e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccec155c00 con 0x7fccec0ca910 2024-04-16T15:41:16.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.666+0000 7fccf0f0e640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fccec1567a0 con 0x7fccec153300 2024-04-16T15:41:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.666+0000 7fccf0f0e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fccec15a030 con 0x7fccec0caf00 2024-04-16T15:41:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.666+0000 7fccea59c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fccec153300 0x7fccec159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.666+0000 7fccead9d640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fccec0ca910 0x7fccec0c0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48198/0 (socket says 172.21.15.71:48198) 2024-04-16T15:41:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.666+0000 7fccead9d640 1 -- 172.21.15.71:0/3972212994 learned_addr learned my addr 172.21.15.71:0/3972212994 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.667+0000 7fccea59c640 1 -- 172.21.15.71:0/3972212994 >> v2:172.21.15.122:3300/0 conn(0x7fccec0caf00 msgr2=0x7fccec0ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.667+0000 7fccea59c640 1 --2- 172.21.15.71:0/3972212994 >> v2:172.21.15.122:3300/0 conn(0x7fccec0caf00 0x7fccec0ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.667+0000 7fccea59c640 1 -- 172.21.15.71:0/3972212994 >> v1:172.21.15.71:6789/0 conn(0x7fccec0ca910 legacy=0x7fccec0c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.667+0000 7fccea59c640 1 -- 172.21.15.71:0/3972212994 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec155e70 con 0x7fccec153300 2024-04-16T15:41:16.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.667+0000 7fccea59c640 1 --2- 172.21.15.71:0/3972212994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec153300 0x7fccec159af0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fccec0aae90 tx=0x7fccdc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4b986368d7b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fcce959a640 1 -- 172.21.15.71:0/3972212994 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0315a0 con 0x7fccec153300 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fcce959a640 1 -- 172.21.15.71:0/3972212994 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccdc037070 con 0x7fccec153300 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fcce959a640 1 -- 172.21.15.71:0/3972212994 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc031960 con 0x7fccec153300 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fccf0f0e640 1 -- 172.21.15.71:0/3972212994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec153300 msgr2=0x7fccec159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/3972212994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec153300 0x7fccec159af0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fccec0aae90 tx=0x7fccdc02b840 comp rx=0 tx=0).stop 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fccf0f0e640 1 -- 172.21.15.71:0/3972212994 shutdown_connections 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/3972212994 >> v2:172.21.15.122:3300/0 conn(0x7fccec0caf00 0x7fccec0ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/3972212994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec153300 0x7fccec159af0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fccf0f0e640 1 -- 172.21.15.71:0/3972212994 >> 172.21.15.71:0/3972212994 conn(0x7fccec0c7f40 msgr2=0x7fccec0c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fccf0f0e640 1 -- 172.21.15.71:0/3972212994 shutdown_connections 2024-04-16T15:41:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.668+0000 7fccf0f0e640 1 -- 172.21.15.71:0/3972212994 wait complete. 2024-04-16T15:41:16.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.670+0000 7fccf0f0e640 1 Processor -- start 2024-04-16T15:41:16.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.670+0000 7fccf0f0e640 1 -- start start 2024-04-16T15:41:16.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fccf0f0e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccec0ca910 0x7fccec155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:16.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fccf0f0e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec0caf00 0x7fccec1edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:16.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fccf0f0e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fccec153300 0x7fccec1ee230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:16.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fccf0f0e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fccec0c0a70 con 0x7fccec0caf00 2024-04-16T15:41:16.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fccf0f0e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fccec156430 con 0x7fccec0ca910 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fccf0f0e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fccec0c0bf0 con 0x7fccec153300 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fcce9d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec0caf00 0x7fccec1edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fcce9d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec0caf00 0x7fccec1edcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57778/0 (socket says 172.21.15.71:57778) 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fcce9d9b640 1 -- 172.21.15.71:0/2440070359 learned_addr learned my addr 172.21.15.71:0/2440070359 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fccea59c640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccec0ca910 0x7fccec155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fccead9d640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fccec153300 0x7fccec1ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fcce9d9b640 1 -- 172.21.15.71:0/2440070359 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fccec153300 msgr2=0x7fccec1ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fcce9d9b640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fccec153300 0x7fccec1ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fcce9d9b640 1 -- 172.21.15.71:0/2440070359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccec0ca910 msgr2=0x7fccec155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fcce9d9b640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccec0ca910 0x7fccec155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.671+0000 7fcce9d9b640 1 -- 172.21.15.71:0/2440070359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec1ed250 con 0x7fccec0caf00 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.672+0000 7fcce9d9b640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec0caf00 0x7fccec1edcf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b4a0 tx=0x7fcce000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.672+0000 7fccdaffd640 1 -- 172.21.15.71:0/2440070359 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0014070 con 0x7fccec0caf00 2024-04-16T15:41:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.672+0000 7fccdaffd640 1 -- 172.21.15.71:0/2440070359 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce0010610 con 0x7fccec0caf00 2024-04-16T15:41:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.672+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec1efbc0 con 0x7fccec0caf00 2024-04-16T15:41:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.672+0000 7fccdaffd640 1 -- 172.21.15.71:0/2440070359 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0010a00 con 0x7fccec0caf00 2024-04-16T15:41:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.672+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec1ed4f0 con 0x7fccec0caf00 2024-04-16T15:41:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.673+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccec068c10 con 0x7fccec0caf00 2024-04-16T15:41:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.673+0000 7fccdaffd640 1 -- 172.21.15.71:0/2440070359 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcce0017070 con 0x7fccec0caf00 2024-04-16T15:41:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.674+0000 7fccdaffd640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcccc06c920 0x7fcccc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.674+0000 7fccdaffd640 1 -- 172.21.15.71:0/2440070359 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7fcce008b280 con 0x7fccec0caf00 2024-04-16T15:41:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.674+0000 7fccea59c640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcccc06c920 0x7fcccc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.674+0000 7fccea59c640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcccc06c920 0x7fcccc06ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fccdc036000 tx=0x7fccdc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:16.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.678+0000 7fccdaffd640 1 -- 172.21.15.71:0/2440070359 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcce0059260 con 0x7fccec0caf00 2024-04-16T15:41:16.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:16.869+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fccec0c6e50 con 0x7fccec0caf00 2024-04-16T15:41:17.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.478+0000 7fccdaffd640 1 -- 172.21.15.71:0/2440070359 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7fcce005d560 con 0x7fccec0caf00 2024-04-16T15:41:17.477 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2024-04-16T15:41:17.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcccc06c920 msgr2=0x7fcccc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcccc06c920 0x7fcccc06ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fccdc036000 tx=0x7fccdc03d040 comp rx=0 tx=0).stop 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec0caf00 msgr2=0x7fccec1edcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec0caf00 0x7fccec1edcf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b4a0 tx=0x7fcce000b960 comp rx=0 tx=0).stop 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 shutdown_connections 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcccc06c920 0x7fcccc06ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fccec153300 0x7fccec1ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccec0caf00 0x7fccec1edcf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 --2- 172.21.15.71:0/2440070359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccec0ca910 0x7fccec155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 >> 172.21.15.71:0/2440070359 conn(0x7fccec0c7f40 msgr2=0x7fccec0c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 shutdown_connections 2024-04-16T15:41:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.482+0000 7fccf0f0e640 1 -- 172.21.15.71:0/2440070359 wait complete. 2024-04-16T15:41:17.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-16T15:41:17.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.605+0000 7fdef65a8640 1 Processor -- start 2024-04-16T15:41:17.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.605+0000 7fdef65a8640 1 -- start start 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdef65a8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdef00c32a0 0x7fdef015e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdef65a8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef00c27c0 con 0x7fdef0155dc0 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdef65a8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef00c2aa0 con 0x7fdef006fac0 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdef65a8640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fdef0073f20 con 0x7fdef00c32a0 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdef4b45640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fdef00c32a0 0x7fdef015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdeeffff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fdef0155dc0 0x7fdef01561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49698/0 (socket says 172.21.15.71:49698) 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdeeffff640 1 -- 172.21.15.71:0/3291743566 learned_addr learned my addr 172.21.15.71:0/3291743566 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdef4b45640 1 -- 172.21.15.71:0/3291743566 >> v1:172.21.15.122:6789/0 conn(0x7fdef006fac0 legacy=0x7fdef00ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdef4b45640 1 -- 172.21.15.71:0/3291743566 >> v1:172.21.15.71:6789/0 conn(0x7fdef0155dc0 legacy=0x7fdef01561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdef4b45640 1 -- 172.21.15.71:0/3291743566 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef00c2d80 con 0x7fdef00c32a0 2024-04-16T15:41:17.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.606+0000 7fdeeeffd640 1 -- 172.21.15.71:0/3291743566 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807580209 0 0) 0x7fdef00c27c0 con 0x7fdef0155dc0 2024-04-16T15:41:17.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.607+0000 7fdef4b45640 1 --2- 172.21.15.71:0/3291743566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef015e160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009d90 tx=0x7fdee802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c67375ee72cf36e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:17.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.607+0000 7fdeeeffd640 1 -- 172.21.15.71:0/3291743566 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8039070 con 0x7fdef00c32a0 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.607+0000 7fdeeeffd640 1 -- 172.21.15.71:0/3291743566 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee802cc60 con 0x7fdef00c32a0 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.607+0000 7fdeeeffd640 1 -- 172.21.15.71:0/3291743566 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8034660 con 0x7fdef00c32a0 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.608+0000 7fdef65a8640 1 -- 172.21.15.71:0/3291743566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 msgr2=0x7fdef015e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.608+0000 7fdef65a8640 1 --2- 172.21.15.71:0/3291743566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef015e160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009d90 tx=0x7fdee802beb0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.608+0000 7fdef65a8640 1 -- 172.21.15.71:0/3291743566 shutdown_connections 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.608+0000 7fdef65a8640 1 --2- 172.21.15.71:0/3291743566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef015e160 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.608+0000 7fdef65a8640 1 -- 172.21.15.71:0/3291743566 >> 172.21.15.71:0/3291743566 conn(0x7fdef00c3e10 msgr2=0x7fdef00c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.608+0000 7fdef65a8640 1 -- 172.21.15.71:0/3291743566 shutdown_connections 2024-04-16T15:41:17.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.608+0000 7fdef65a8640 1 -- 172.21.15.71:0/3291743566 wait complete. 2024-04-16T15:41:17.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.609+0000 7fdef65a8640 1 Processor -- start 2024-04-16T15:41:17.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.609+0000 7fdef65a8640 1 -- start start 2024-04-16T15:41:17.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef65a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef01ea690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef65a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdef0155dc0 0x7fdef01457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef65a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdef0145f50 0x7fdef0140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef65a8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdef00c18b0 con 0x7fdef00c32a0 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef65a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0156ab0 con 0x7fdef0145f50 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef65a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdef006bd70 con 0x7fdef0155dc0 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef01ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef01ea690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58042/0 (socket says 172.21.15.71:58042) 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 -- 172.21.15.71:0/3133606162 learned_addr learned my addr 172.21.15.71:0/3133606162 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef4b45640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdef0145f50 0x7fdef0140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeef7fe640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdef0155dc0 0x7fdef01457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 -- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdef0155dc0 msgr2=0x7fdef01457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdef0155dc0 0x7fdef01457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 -- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdef0145f50 msgr2=0x7fdef0140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdef0145f50 0x7fdef0140990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 -- 172.21.15.71:0/3133606162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef0145ce0 con 0x7fdef00c32a0 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef4b45640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdef0145f50 0x7fdef0140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeef7fe640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdef0155dc0 0x7fdef01457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:17.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeeffff640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef01ea690 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c460 tx=0x7fdee000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeecff9640 1 -- 172.21.15.71:0/3133606162 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0018070 con 0x7fdef00c32a0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeecff9640 1 -- 172.21.15.71:0/3133606162 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee0007500 con 0x7fdef00c32a0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef0146710 con 0x7fdef00c32a0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdeecff9640 1 -- 172.21.15.71:0/3133606162 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0004a20 con 0x7fdef00c32a0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.610+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef01e9bb0 con 0x7fdef00c32a0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.611+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef0068d20 con 0x7fdef00c32a0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.612+0000 7fdeecff9640 1 -- 172.21.15.71:0/3133606162 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdee001b070 con 0x7fdef00c32a0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.612+0000 7fdeecff9640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdedc06c8f0 0x7fdedc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.612+0000 7fdeef7fe640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdedc06c8f0 0x7fdedc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.612+0000 7fdeecff9640 1 -- 172.21.15.71:0/3133606162 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fdee008ab50 con 0x7fdef00c32a0 2024-04-16T15:41:17.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.613+0000 7fdeef7fe640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdedc06c8f0 0x7fdedc06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdee4007fd0 tx=0x7fdee4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:17.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.617+0000 7fdeecff9640 1 -- 172.21.15.71:0/3133606162 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdee00591d0 con 0x7fdef00c32a0 2024-04-16T15:41:17.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.861+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdef00c2120 con 0x7fdef00c32a0 2024-04-16T15:41:17.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.861+0000 7fdeecff9640 1 -- 172.21.15.71:0/3133606162 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6065 (secure 0 0 0) 0x7fdee0058ad0 con 0x7fdef00c32a0 2024-04-16T15:41:17.860 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:17.860 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCtmx5maSbSOhAAEIlR48PaHF+TNphBw608dQ==", 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.861 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCumx5mYc8lDBAAkxIO7XDmrv0CRfaWHUGQ6A==", 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCumx5mPxjpBhAAgnn0yvcH+OHJGkmM+wREeQ==", 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.862 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCumx5meQ/JExAAvSY3b7bj83gzM9ghd9Qkrg==", 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCxmx5mEtyQKhAAvTQwwrkkdepLFbdaJ3z6+w==", 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQC0mx5mizaHKRAA9ods0Zji7MgKtrqv41vTtg==", 2024-04-16T15:41:17.864 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQC3mx5mXzwKKBAANUOg3ynvzO62gVGupvq0kw==", 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQC6mx5mqNvLJhAA3Qj3Joh87KtLHV9xbJ7vWA==", 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDAmx5mb77YLxAAB1NQ96N0oEq8guwEeOcSbQ==", 2024-04-16T15:41:17.866 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDDmx5mabyhLhAA0vq9zxx4L1s0w6vLSADfRw==", 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDGmx5mVKFQLRAAUdDofDpmWZykpPbufPrVVw==", 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDJmx5m7V13KxAAb+B6sdFAwn/o48wloWR0sg==", 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2024-04-16T15:41:17.868 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCumx5me0E3GBAAUrYP3hRI128xd+yf1A5VaA==", 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.869 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCumx5m++vGGxAAkTQKAgscEljR9aCDLR3Oww==", 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCtmx5mOYiJHBAAZo50L/i7QBkrFwWeteL+Ig==", 2024-04-16T15:41:17.870 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-mds", 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDPmx5msWLmGxAANGnNgQiT4uB976FVLtksbA==", 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-mds" 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.871 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-mgr", 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDPmx5mcmvmGxAAo5w7hw0gKSeHDlVwR5hb9A==", 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-osd", 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDPmx5mj3LmGxAA43BO/oEwsohtPU+GYzEaUg==", 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-osd" 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rbd", 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDPmx5mZXnmGxAAbOEpFEaoNaQ/hAGLustCqw==", 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDPmx5mWYDmGxAALyVwBMaXFHeyjNp5j81w9g==", 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.873 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rgw", 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQDPmx5mXIfmGxAAw8TMVN7vo82zQ8pNznT9jQ==", 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCtmx5mwREuNxAA0isz1n+5hfl4bBvzO3UOuA==", 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQCtmx5mmB1aMhAAFQuNW0KOn1oJcsNCxMKeXQ==", 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-04-16T15:41:17.875 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdedc06c8f0 msgr2=0x7fdedc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdedc06c8f0 0x7fdedc06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdee4007fd0 tx=0x7fdee4009040 comp rx=0 tx=0).stop 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 msgr2=0x7fdef01ea690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef01ea690 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c460 tx=0x7fdee000c920 comp rx=0 tx=0).stop 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 shutdown_connections 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fdedc06c8f0 0x7fdedc06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdef0145f50 0x7fdef0140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdef0155dc0 0x7fdef01457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 --2- 172.21.15.71:0/3133606162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdef00c32a0 0x7fdef01ea690 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.863+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 >> 172.21.15.71:0/3133606162 conn(0x7fdef00c3e10 msgr2=0x7fdef0146f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:17.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.864+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 shutdown_connections 2024-04-16T15:41:17.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.864+0000 7fdef65a8640 1 -- 172.21.15.71:0/3133606162 wait complete. 2024-04-16T15:41:17.878 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-16T15:41:17.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.984+0000 7ff70ba37640 1 Processor -- start 2024-04-16T15:41:17.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.984+0000 7ff70ba37640 1 -- start start 2024-04-16T15:41:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff70ba37640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff7040c56b0 0x7ff704071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff70ba37640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff7040c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff70ba37640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff704071fe0 con 0x7ff7040c66e0 2024-04-16T15:41:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff70ba37640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff704159290 con 0x7ff7040c56b0 2024-04-16T15:41:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff70ba37640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff704156140 con 0x7ff704155c10 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff709fd4640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7ff7040c66e0 0x7ff704159720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49706/0 (socket says 172.21.15.71:49706) 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff709fd4640 1 -- 172.21.15.71:0/1564475941 learned_addr learned my addr 172.21.15.71:0/1564475941 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff7097d3640 1 --2- 172.21.15.71:0/1564475941 >> v2:172.21.15.122:3300/0 conn(0x7ff7040c56b0 0x7ff704071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff708fd2640 1 --2- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff7040c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff6fbfff640 1 -- 172.21.15.71:0/1564475941 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 669508737 0 0) 0x7ff704071fe0 con 0x7ff7040c66e0 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff6fbfff640 1 -- 172.21.15.71:0/1564475941 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6f0003520 con 0x7ff7040c66e0 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff7097d3640 1 -- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 msgr2=0x7ff7040c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff7097d3640 1 --2- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff7040c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff7097d3640 1 -- 172.21.15.71:0/1564475941 >> v1:172.21.15.71:6789/0 conn(0x7ff7040c66e0 legacy=0x7ff704159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff6fbfff640 1 -- 172.21.15.71:0/1564475941 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2975154273 0 0) 0x7ff6f0003520 con 0x7ff7040c66e0 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff7097d3640 1 -- 172.21.15.71:0/1564475941 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff704156420 con 0x7ff7040c56b0 2024-04-16T15:41:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.985+0000 7ff708fd2640 1 --2- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff7040c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.986+0000 7ff7097d3640 1 --2- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 0x7ff704071aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff7000097a0 tx=0x7ff70002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5cfd88049e1d2dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.986+0000 7ff6fbfff640 1 -- 172.21.15.71:0/1564475941 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700037070 con 0x7ff7040c56b0 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.986+0000 7ff6fbfff640 1 -- 172.21.15.71:0/1564475941 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff700030510 con 0x7ff7040c56b0 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.986+0000 7ff6fbfff640 1 -- 172.21.15.71:0/1564475941 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0019f0 con 0x7ff7040c56b0 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.987+0000 7ff70ba37640 1 -- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 msgr2=0x7ff704071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.987+0000 7ff70ba37640 1 --2- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 0x7ff704071aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff7000097a0 tx=0x7ff70002b790 comp rx=0 tx=0).stop 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.987+0000 7ff70ba37640 1 -- 172.21.15.71:0/1564475941 shutdown_connections 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.987+0000 7ff70ba37640 1 --2- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff7040c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.987+0000 7ff70ba37640 1 --2- 172.21.15.71:0/1564475941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 0x7ff704071aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.987+0000 7ff70ba37640 1 -- 172.21.15.71:0/1564475941 >> 172.21.15.71:0/1564475941 conn(0x7ff7040ab440 msgr2=0x7ff7040ab850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.987+0000 7ff70ba37640 1 -- 172.21.15.71:0/1564475941 shutdown_connections 2024-04-16T15:41:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.987+0000 7ff70ba37640 1 -- 172.21.15.71:0/1564475941 wait complete. 2024-04-16T15:41:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.988+0000 7ff70ba37640 1 Processor -- start 2024-04-16T15:41:17.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.988+0000 7ff70ba37640 1 -- start start 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff70ba37640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 0x7ff7041526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff70ba37640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7040c66e0 0x7ff704151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff70ba37640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff704151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff70ba37640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff7040c5a90 con 0x7ff7040c66e0 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff70ba37640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041569e0 con 0x7ff7040c56b0 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff70ba37640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff7040c75b0 con 0x7ff704155c10 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7040c66e0 0x7ff704151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7040c66e0 0x7ff704151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58056/0 (socket says 172.21.15.71:58056) 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 -- 172.21.15.71:0/4149047644 learned_addr learned my addr 172.21.15.71:0/4149047644 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff709fd4640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff704151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff7097d3640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 0x7ff7041526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 -- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 msgr2=0x7ff704151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff704151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 -- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 msgr2=0x7ff7041526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 0x7ff7041526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 -- 172.21.15.71:0/4149047644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7040c8580 con 0x7ff7040c66e0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff709fd4640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff704151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.989+0000 7ff708fd2640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7040c66e0 0x7ff704151350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400e910 tx=0x7ff6f400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.990+0000 7ff6f9ffb640 1 -- 172.21.15.71:0/4149047644 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4017070 con 0x7ff7040c66e0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.990+0000 7ff6f9ffb640 1 -- 172.21.15.71:0/4149047644 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f400ca00 con 0x7ff7040c66e0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.990+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7040ae440 con 0x7ff7040c66e0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.990+0000 7ff6f9ffb640 1 -- 172.21.15.71:0/4149047644 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400cdb0 con 0x7ff7040c66e0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.990+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff704150430 con 0x7ff7040c66e0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.991+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff704068c10 con 0x7ff7040c66e0 2024-04-16T15:41:17.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.991+0000 7ff6f9ffb640 1 -- 172.21.15.71:0/4149047644 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6f401a070 con 0x7ff7040c66e0 2024-04-16T15:41:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.992+0000 7ff6f9ffb640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6e806c8f0 0x7ff6e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.992+0000 7ff6f9ffb640 1 -- 172.21.15.71:0/4149047644 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7ff6f408b7b0 con 0x7ff7040c66e0 2024-04-16T15:41:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.992+0000 7ff7097d3640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6e806c8f0 0x7ff6e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:17.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.992+0000 7ff7097d3640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6e806c8f0 0x7ff6e806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff700034000 tx=0x7ff70002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:17.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:17.996+0000 7ff6f9ffb640 1 -- 172.21.15.71:0/4149047644 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6f4059e30 con 0x7ff7040c66e0 2024-04-16T15:41:18.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.241+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ff704071880 con 0x7ff7040c66e0 2024-04-16T15:41:18.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.243+0000 7ff6f9ffb640 1 -- 172.21.15.71:0/4149047644 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7ff6f4059730 con 0x7ff7040c66e0 2024-04-16T15:41:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6e806c8f0 msgr2=0x7ff6e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6e806c8f0 0x7ff6e806edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff700034000 tx=0x7ff70002f000 comp rx=0 tx=0).stop 2024-04-16T15:41:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7040c66e0 msgr2=0x7ff704151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7040c66e0 0x7ff704151350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400e910 tx=0x7ff6f400edd0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 shutdown_connections 2024-04-16T15:41:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ff6e806c8f0 0x7ff6e806edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff704155c10 0x7ff704151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7040c66e0 0x7ff704151350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 --2- 172.21.15.71:0/4149047644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7040c56b0 0x7ff7041526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 >> 172.21.15.71:0/4149047644 conn(0x7ff7040ab440 msgr2=0x7ff704159620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 shutdown_connections 2024-04-16T15:41:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.245+0000 7ff70ba37640 1 -- 172.21.15.71:0/4149047644 wait complete. 2024-04-16T15:41:18.254 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-16T15:41:18.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.360+0000 7f52c5f1c640 1 Processor -- start 2024-04-16T15:41:18.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.360+0000 7f52c5f1c640 1 -- start start 2024-04-16T15:41:18.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.360+0000 7f52c5f1c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c014f9f0 0x7f52c00aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.360+0000 7f52c5f1c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f52c00c3500 0x7f52c00c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.360+0000 7f52c5f1c640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c00c2250 con 0x7f52c006c2f0 2024-04-16T15:41:18.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.360+0000 7f52c5f1c640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f52c014fdd0 con 0x7f52c00c3500 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.360+0000 7f52c5f1c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f52c0152bc0 con 0x7f52c014f9f0 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52beffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f52c00c3500 0x7f52c00c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52beffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f52c00c3500 0x7f52c00c1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58062/0 (socket says 172.21.15.71:58062) 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52bffff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f52c006c2f0 0x7f52c006f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49710/0 (socket says 172.21.15.71:49710) 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52beffd640 1 -- 172.21.15.71:0/2512271242 learned_addr learned my addr 172.21.15.71:0/2512271242 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52bf7fe640 1 --2- 172.21.15.71:0/2512271242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c014f9f0 0x7f52c00aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52beffd640 1 -- 172.21.15.71:0/2512271242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c014f9f0 msgr2=0x7f52c00aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52beffd640 1 --2- 172.21.15.71:0/2512271242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c014f9f0 0x7f52c00aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52beffd640 1 -- 172.21.15.71:0/2512271242 >> v1:172.21.15.71:6789/0 conn(0x7f52c006c2f0 legacy=0x7f52c006f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52beffd640 1 -- 172.21.15.71:0/2512271242 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c0152ea0 con 0x7f52c00c3500 2024-04-16T15:41:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52be7fc640 1 -- 172.21.15.71:0/2512271242 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993584147 0 0) 0x7f52c00c2250 con 0x7f52c006c2f0 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.361+0000 7f52bf7fe640 1 --2- 172.21.15.71:0/2512271242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c014f9f0 0x7f52c00aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52beffd640 1 --2- 172.21.15.71:0/2512271242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c00c1d10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f52b400bb10 tx=0x7f52b402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccb26b1c787ab3c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52be7fc640 1 -- 172.21.15.71:0/2512271242 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b403b070 con 0x7f52c00c3500 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52be7fc640 1 -- 172.21.15.71:0/2512271242 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b4036440 con 0x7f52c00c3500 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52be7fc640 1 -- 172.21.15.71:0/2512271242 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4036810 con 0x7f52c00c3500 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52c5f1c640 1 -- 172.21.15.71:0/2512271242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 msgr2=0x7f52c00c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/2512271242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c00c1d10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f52b400bb10 tx=0x7f52b402e0c0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52c5f1c640 1 -- 172.21.15.71:0/2512271242 shutdown_connections 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/2512271242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c00c1d10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/2512271242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c014f9f0 0x7f52c00aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52c5f1c640 1 -- 172.21.15.71:0/2512271242 >> 172.21.15.71:0/2512271242 conn(0x7f52c006e220 msgr2=0x7f52c00c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.362+0000 7f52c5f1c640 1 -- 172.21.15.71:0/2512271242 shutdown_connections 2024-04-16T15:41:18.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.363+0000 7f52c5f1c640 1 -- 172.21.15.71:0/2512271242 wait complete. 2024-04-16T15:41:18.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52c5f1c640 1 Processor -- start 2024-04-16T15:41:18.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52c5f1c640 1 -- start start 2024-04-16T15:41:18.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52c5f1c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c006c2f0 0x7f52c0150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52c5f1c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c01da490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52c5f1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c014f9f0 0x7f52c01da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52c5f1c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f52c006f780 con 0x7f52c00c3500 2024-04-16T15:41:18.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52c5f1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52c006bdd0 con 0x7f52c014f9f0 2024-04-16T15:41:18.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52c5f1c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f52c006f900 con 0x7f52c006c2f0 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.364+0000 7f52beffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c01da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52beffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c01da490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58072/0 (socket says 172.21.15.71:58072) 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52beffd640 1 -- 172.21.15.71:0/4120261911 learned_addr learned my addr 172.21.15.71:0/4120261911 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52bf7fe640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c006c2f0 0x7f52c0150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52bffff640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c014f9f0 0x7f52c01da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52beffd640 1 -- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c006c2f0 msgr2=0x7f52c0150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52beffd640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c006c2f0 0x7f52c0150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52beffd640 1 -- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c014f9f0 msgr2=0x7f52c01da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52beffd640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c014f9f0 0x7f52c01da9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52beffd640 1 -- 172.21.15.71:0/4120261911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c01e7110 con 0x7f52c00c3500 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52bffff640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c014f9f0 0x7f52c01da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52beffd640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c01da490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f52b4039040 tx=0x7f52b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52a7fff640 1 -- 172.21.15.71:0/4120261911 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b403b040 con 0x7f52c00c3500 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52a7fff640 1 -- 172.21.15.71:0/4120261911 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b4004230 con 0x7f52c00c3500 2024-04-16T15:41:18.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c00c17d0 con 0x7f52c00c3500 2024-04-16T15:41:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52a7fff640 1 -- 172.21.15.71:0/4120261911 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4003610 con 0x7f52c00c3500 2024-04-16T15:41:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.365+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c01e76b0 con 0x7f52c00c3500 2024-04-16T15:41:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.366+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c00aa980 con 0x7f52c00c3500 2024-04-16T15:41:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.367+0000 7f52a7fff640 1 -- 172.21.15.71:0/4120261911 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52b40037b0 con 0x7f52c00c3500 2024-04-16T15:41:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.367+0000 7f52a7fff640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52ac06c8f0 0x7f52ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.367+0000 7f52a7fff640 1 -- 172.21.15.71:0/4120261911 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f52b40b0b20 con 0x7f52c00c3500 2024-04-16T15:41:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.367+0000 7f52bf7fe640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52ac06c8f0 0x7f52ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.367+0000 7f52bf7fe640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52ac06c8f0 0x7f52ac06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f52b000a950 tx=0x7f52b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:18.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.371+0000 7f52a7fff640 1 -- 172.21.15.71:0/4120261911 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52c00aa980 con 0x7f52c00c3500 2024-04-16T15:41:18.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.620+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f52c0073480 con 0x7f52c00c3500 2024-04-16T15:41:18.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.622+0000 7f52a7fff640 1 -- 172.21.15.71:0/4120261911 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f52b407f1a0 con 0x7f52c00c3500 2024-04-16T15:41:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52ac06c8f0 msgr2=0x7f52ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52ac06c8f0 0x7f52ac06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f52b000a950 tx=0x7f52b0008040 comp rx=0 tx=0).stop 2024-04-16T15:41:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 msgr2=0x7f52c01da490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c01da490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f52b4039040 tx=0x7f52b40046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 shutdown_connections 2024-04-16T15:41:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f52ac06c8f0 0x7f52ac06edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c014f9f0 0x7f52c01da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c00c3500 0x7f52c01da490 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 --2- 172.21.15.71:0/4120261911 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52c006c2f0 0x7f52c0150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 >> 172.21.15.71:0/4120261911 conn(0x7f52c006e220 msgr2=0x7f52c014eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 shutdown_connections 2024-04-16T15:41:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.624+0000 7f52c5f1c640 1 -- 172.21.15.71:0/4120261911 wait complete. 2024-04-16T15:41:18.634 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-16T15:41:18.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.739+0000 7ffb9842a640 1 Processor -- start 2024-04-16T15:41:18.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.739+0000 7ffb9842a640 1 -- start start 2024-04-16T15:41:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb9842a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 0x7ffb9006deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb9842a640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7ffb9006fae0 0x7ffb9006ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb9842a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb90074010 con 0x7ffb9014ca80 2024-04-16T15:41:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb9842a640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7ffb900c3540 con 0x7ffb9006fae0 2024-04-16T15:41:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb9842a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ffb90073a90 con 0x7ffb9006da50 2024-04-16T15:41:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb969c7640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7ffb9006fae0 0x7ffb9006ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb969c7640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7ffb9006fae0 0x7ffb9006ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58080/0 (socket says 172.21.15.71:58080) 2024-04-16T15:41:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb969c7640 1 -- 172.21.15.71:0/518612747 learned_addr learned my addr 172.21.15.71:0/518612747 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb959c5640 1 --2- 172.21.15.71:0/518612747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 0x7ffb9006deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb969c7640 1 -- 172.21.15.71:0/518612747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 msgr2=0x7ffb9006deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb969c7640 1 --2- 172.21.15.71:0/518612747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 0x7ffb9006deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb969c7640 1 -- 172.21.15.71:0/518612747 >> v1:172.21.15.122:6789/0 conn(0x7ffb9014ca80 legacy=0x7ffb90153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.740+0000 7ffb969c7640 1 -- 172.21.15.71:0/518612747 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb900c2f30 con 0x7ffb9006fae0 2024-04-16T15:41:18.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.741+0000 7ffb969c7640 1 --2- 172.21.15.71:0/518612747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9006fae0 0x7ffb9006ff00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800d150 tx=0x7ffb8802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbbcb0bf132a547 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:18.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.741+0000 7ffb951c4640 1 -- 172.21.15.71:0/518612747 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8802e9e0 con 0x7ffb9006fae0 2024-04-16T15:41:18.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.741+0000 7ffb951c4640 1 -- 172.21.15.71:0/518612747 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8802eb80 con 0x7ffb9006fae0 2024-04-16T15:41:18.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.741+0000 7ffb951c4640 1 -- 172.21.15.71:0/518612747 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88009600 con 0x7ffb9006fae0 2024-04-16T15:41:18.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.742+0000 7ffb9842a640 1 -- 172.21.15.71:0/518612747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9006fae0 msgr2=0x7ffb9006ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.742+0000 7ffb9842a640 1 --2- 172.21.15.71:0/518612747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9006fae0 0x7ffb9006ff00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800d150 tx=0x7ffb8802e040 comp rx=0 tx=0).stop 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.742+0000 7ffb9842a640 1 -- 172.21.15.71:0/518612747 shutdown_connections 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.742+0000 7ffb9842a640 1 --2- 172.21.15.71:0/518612747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9006fae0 0x7ffb9006ff00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.742+0000 7ffb9842a640 1 --2- 172.21.15.71:0/518612747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 0x7ffb9006deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.742+0000 7ffb9842a640 1 -- 172.21.15.71:0/518612747 >> 172.21.15.71:0/518612747 conn(0x7ffb900729d0 msgr2=0x7ffb90072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.742+0000 7ffb9842a640 1 -- 172.21.15.71:0/518612747 shutdown_connections 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.742+0000 7ffb9842a640 1 -- 172.21.15.71:0/518612747 wait complete. 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.743+0000 7ffb9842a640 1 Processor -- start 2024-04-16T15:41:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.743+0000 7ffb9842a640 1 -- start start 2024-04-16T15:41:18.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.743+0000 7ffb9842a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 0x7ffb901ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.743+0000 7ffb9842a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb9006fae0 0x7ffb900c1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.743+0000 7ffb9842a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9014ca80 0x7ffb901513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.743+0000 7ffb9842a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9014ce60 con 0x7ffb9014ca80 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.743+0000 7ffb9842a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffb900c31d0 con 0x7ffb9006fae0 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.743+0000 7ffb9842a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9014cfe0 con 0x7ffb9006da50 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9014ca80 0x7ffb901513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9014ca80 0x7ffb901513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58090/0 (socket says 172.21.15.71:58090) 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 -- 172.21.15.71:0/2652857490 learned_addr learned my addr 172.21.15.71:0/2652857490 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb959c5640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb9006fae0 0x7ffb900c1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb961c6640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 0x7ffb901ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 -- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 msgr2=0x7ffb901ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 0x7ffb901ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 -- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb9006fae0 msgr2=0x7ffb900c1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb9006fae0 0x7ffb900c1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 -- 172.21.15.71:0/2652857490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb90150f80 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb959c5640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb9006fae0 0x7ffb900c1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb969c7640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9014ca80 0x7ffb901513a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffb88038040 tx=0x7ffb88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb86ffd640 1 -- 172.21.15.71:0/2652857490 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8802e9e0 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb86ffd640 1 -- 172.21.15.71:0/2652857490 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb88045070 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb86ffd640 1 -- 172.21.15.71:0/2652857490 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88036a60 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb900c6410 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.744+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb900c69d0 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.745+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb90068c10 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.746+0000 7ffb86ffd640 1 -- 172.21.15.71:0/2652857490 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb88036c00 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.746+0000 7ffb86ffd640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffb7406c8f0 0x7ffb7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.746+0000 7ffb86ffd640 1 -- 172.21.15.71:0/2652857490 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7ffb880ae9a0 con 0x7ffb9014ca80 2024-04-16T15:41:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.746+0000 7ffb961c6640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffb7406c8f0 0x7ffb7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:18.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.747+0000 7ffb961c6640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffb7406c8f0 0x7ffb7406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c000f60 tx=0x7ffb8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:18.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.750+0000 7ffb86ffd640 1 -- 172.21.15.71:0/2652857490 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb8807d0c0 con 0x7ffb9014ca80 2024-04-16T15:41:18.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.991+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7ffb9014c0e0 con 0x7ffb9014ca80 2024-04-16T15:41:18.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.995+0000 7ffb86ffd640 1 -- 172.21.15.71:0/2652857490 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7ffb88009370 con 0x7ffb9014ca80 2024-04-16T15:41:18.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffb7406c8f0 msgr2=0x7ffb7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffb7406c8f0 0x7ffb7406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c000f60 tx=0x7ffb8c009040 comp rx=0 tx=0).stop 2024-04-16T15:41:18.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9014ca80 msgr2=0x7ffb901513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:18.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9014ca80 0x7ffb901513a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffb88038040 tx=0x7ffb88004a40 comp rx=0 tx=0).stop 2024-04-16T15:41:18.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 shutdown_connections 2024-04-16T15:41:18.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7ffb7406c8f0 0x7ffb7406edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb9014ca80 0x7ffb901513a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb9006fae0 0x7ffb900c1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 --2- 172.21.15.71:0/2652857490 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ffb9006da50 0x7ffb901ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:18.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 >> 172.21.15.71:0/2652857490 conn(0x7ffb900729d0 msgr2=0x7ffb90148080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:18.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 shutdown_connections 2024-04-16T15:41:18.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:18.997+0000 7ffb9842a640 1 -- 172.21.15.71:0/2652857490 wait complete. 2024-04-16T15:41:19.007 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-16T15:41:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.111+0000 7fcd4864a640 1 Processor -- start 2024-04-16T15:41:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.111+0000 7fcd4864a640 1 -- start start 2024-04-16T15:41:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.112+0000 7fcd4864a640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcd4014a480 0x7fcd40072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.112+0000 7fcd4864a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd40072ea0 0x7fcd400a9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.112+0000 7fcd4864a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd40073740 con 0x7fcd40153a60 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.112+0000 7fcd4864a640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fcd4006db90 con 0x7fcd4014a480 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.112+0000 7fcd4864a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4006cf80 con 0x7fcd40072ea0 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.113+0000 7fcd463e6640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcd4014a480 0x7fcd40072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.113+0000 7fcd463e6640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fcd4014a480 0x7fcd40072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58102/0 (socket says 172.21.15.71:58102) 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.113+0000 7fcd463e6640 1 -- 172.21.15.71:0/974121692 learned_addr learned my addr 172.21.15.71:0/974121692 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.113+0000 7fcd45be5640 1 --2- 172.21.15.71:0/974121692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd40072ea0 0x7fcd400a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.113+0000 7fcd463e6640 1 -- 172.21.15.71:0/974121692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd40072ea0 msgr2=0x7fcd400a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.113+0000 7fcd463e6640 1 --2- 172.21.15.71:0/974121692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd40072ea0 0x7fcd400a9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.113+0000 7fcd463e6640 1 -- 172.21.15.71:0/974121692 >> v1:172.21.15.122:6789/0 conn(0x7fcd40153a60 legacy=0x7fcd400a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.113+0000 7fcd463e6640 1 -- 172.21.15.71:0/974121692 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4006d260 con 0x7fcd4014a480 2024-04-16T15:41:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd463e6640 1 --2- 172.21.15.71:0/974121692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd4014a480 0x7fcd40072960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009e50 tx=0x7fcd3c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d072aca2ef97478 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd453e4640 1 -- 172.21.15.71:0/974121692 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c02cb00 con 0x7fcd4014a480 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd453e4640 1 -- 172.21.15.71:0/974121692 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c02cca0 con 0x7fcd4014a480 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd453e4640 1 -- 172.21.15.71:0/974121692 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c034610 con 0x7fcd4014a480 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd4864a640 1 -- 172.21.15.71:0/974121692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd4014a480 msgr2=0x7fcd40072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd4864a640 1 --2- 172.21.15.71:0/974121692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd4014a480 0x7fcd40072960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009e50 tx=0x7fcd3c02bf60 comp rx=0 tx=0).stop 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd4864a640 1 -- 172.21.15.71:0/974121692 shutdown_connections 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd4864a640 1 --2- 172.21.15.71:0/974121692 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd40072ea0 0x7fcd400a9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd4864a640 1 --2- 172.21.15.71:0/974121692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd4014a480 0x7fcd40072960 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd4864a640 1 -- 172.21.15.71:0/974121692 >> 172.21.15.71:0/974121692 conn(0x7fcd4006e6e0 msgr2=0x7fcd4006eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd4864a640 1 -- 172.21.15.71:0/974121692 shutdown_connections 2024-04-16T15:41:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.114+0000 7fcd4864a640 1 -- 172.21.15.71:0/974121692 wait complete. 2024-04-16T15:41:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd4864a640 1 Processor -- start 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd4864a640 1 -- start start 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd4864a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd40072ea0 0x7fcd401454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd4864a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd4014a480 0x7fcd40146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd4864a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40153a60 0x7fcd40146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd4864a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd40073f80 con 0x7fcd40072ea0 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd4864a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4006d820 con 0x7fcd40153a60 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd4864a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcd400aa180 con 0x7fcd4014a480 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd463e6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd40072ea0 0x7fcd401454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd463e6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd40072ea0 0x7fcd401454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58110/0 (socket says 172.21.15.71:58110) 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.116+0000 7fcd463e6640 1 -- 172.21.15.71:0/3594919756 learned_addr learned my addr 172.21.15.71:0/3594919756 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd463e6640 1 -- 172.21.15.71:0/3594919756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd4014a480 msgr2=0x7fcd40146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd463e6640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd4014a480 0x7fcd40146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd463e6640 1 -- 172.21.15.71:0/3594919756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40153a60 msgr2=0x7fcd40146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd46be7640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40153a60 0x7fcd40146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd463e6640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40153a60 0x7fcd40146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd463e6640 1 -- 172.21.15.71:0/3594919756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd400c6300 con 0x7fcd40072ea0 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd46be7640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40153a60 0x7fcd40146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd463e6640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd40072ea0 0x7fcd401454a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c000f80 tx=0x7fcd3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd36ffd640 1 -- 172.21.15.71:0/3594919756 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c043020 con 0x7fcd40072ea0 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd36ffd640 1 -- 172.21.15.71:0/3594919756 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c02ce50 con 0x7fcd40072ea0 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd36ffd640 1 -- 172.21.15.71:0/3594919756 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c009510 con 0x7fcd40072ea0 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd40145e90 con 0x7fcd40072ea0 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.117+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4014dd50 con 0x7fcd40072ea0 2024-04-16T15:41:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.118+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd400c6970 con 0x7fcd40072ea0 2024-04-16T15:41:19.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.118+0000 7fcd36ffd640 1 -- 172.21.15.71:0/3594919756 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd3c046020 con 0x7fcd40072ea0 2024-04-16T15:41:19.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.119+0000 7fcd36ffd640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd2406c8f0 0x7fcd2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.119+0000 7fcd36ffd640 1 -- 172.21.15.71:0/3594919756 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fcd3c0adfd0 con 0x7fcd40072ea0 2024-04-16T15:41:19.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.119+0000 7fcd45be5640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd2406c8f0 0x7fcd2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.119+0000 7fcd45be5640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd2406c8f0 0x7fcd2406edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcd30006a90 tx=0x7fcd30006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.123+0000 7fcd36ffd640 1 -- 172.21.15.71:0/3594919756 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd3c04d030 con 0x7fcd40072ea0 2024-04-16T15:41:19.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.365+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fcd400c6f90 con 0x7fcd40072ea0 2024-04-16T15:41:19.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.367+0000 7fcd36ffd640 1 -- 172.21.15.71:0/3594919756 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fcd400c6f90 con 0x7fcd40072ea0 2024-04-16T15:41:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd2406c8f0 msgr2=0x7fcd2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd2406c8f0 0x7fcd2406edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcd30006a90 tx=0x7fcd30006a20 comp rx=0 tx=0).stop 2024-04-16T15:41:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd40072ea0 msgr2=0x7fcd401454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd40072ea0 0x7fcd401454a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c000f80 tx=0x7fcd3c004a40 comp rx=0 tx=0).stop 2024-04-16T15:41:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 shutdown_connections 2024-04-16T15:41:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fcd2406c8f0 0x7fcd2406edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40153a60 0x7fcd40146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcd4014a480 0x7fcd40146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 --2- 172.21.15.71:0/3594919756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd40072ea0 0x7fcd401454a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 >> 172.21.15.71:0/3594919756 conn(0x7fcd4006e6e0 msgr2=0x7fcd401dd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 shutdown_connections 2024-04-16T15:41:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.369+0000 7fcd4864a640 1 -- 172.21.15.71:0/3594919756 wait complete. 2024-04-16T15:41:19.379 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-16T15:41:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.485+0000 7f242b7bd640 1 Processor -- start 2024-04-16T15:41:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.485+0000 7f242b7bd640 1 -- start start 2024-04-16T15:41:19.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f242b7bd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2424154360 0x7f24240c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f242b7bd640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24241548b0 con 0x7f24240c48d0 2024-04-16T15:41:19.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f242b7bd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24240c7310 con 0x7f2424071250 2024-04-16T15:41:19.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f242b7bd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f24240c75b0 con 0x7f2424154360 2024-04-16T15:41:19.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f2429d5a640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f24240c48d0 0x7f242415e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49712/0 (socket says 172.21.15.71:49712) 2024-04-16T15:41:19.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f2429d5a640 1 -- 172.21.15.71:0/3895478633 learned_addr learned my addr 172.21.15.71:0/3895478633 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:19.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f2428d58640 1 --2- 172.21.15.71:0/3895478633 >> v2:172.21.15.122:3300/0 conn(0x7f2424154360 0x7f24240c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451942564 0 0) 0x7f24241548b0 con 0x7f24240c48d0 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.486+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f240c003520 con 0x7f24240c48d0 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381970082 0 0) 0x7f24240c7310 con 0x7f2424071250 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24241548b0 con 0x7f2424071250 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 851091939 0 0) 0x7f240c003520 con 0x7f24240c48d0 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 >> v2:172.21.15.122:3300/0 conn(0x7f2424154360 msgr2=0x7f24240c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 --2- 172.21.15.71:0/3895478633 >> v2:172.21.15.122:3300/0 conn(0x7f2424154360 0x7f24240c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 >> v1:172.21.15.122:6789/0 conn(0x7f2424071250 legacy=0x7f2424071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240c7890 con 0x7f24240c48d0 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f24140019f0 con 0x7f24240c48d0 2024-04-16T15:41:19.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 >> v1:172.21.15.71:6789/0 conn(0x7f24240c48d0 legacy=0x7f242415e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f240c003b90 0x7f240c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c004800 0x7f240c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f240c005210 0x7f240c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f24240c75b0 con 0x7f240c005210 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f240c00c140 con 0x7f240c003b90 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f240c00c340 con 0x7f240c004800 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2429d5a640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f240c005210 0x7f240c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2429559640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f240c003b90 0x7f240c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2428d58640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c004800 0x7f240c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2429d5a640 1 -- 172.21.15.71:0/3895478633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c004800 msgr2=0x7f240c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2429d5a640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c004800 0x7f240c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2429d5a640 1 -- 172.21.15.71:0/3895478633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f240c003b90 msgr2=0x7f240c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2429d5a640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f240c003b90 0x7f240c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.487+0000 7f2429d5a640 1 -- 172.21.15.71:0/3895478633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c003520 con 0x7f240c005210 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.488+0000 7f2429d5a640 1 -- 172.21.15.71:0/3895478633 reap_dead start 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.488+0000 7f2429d5a640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f240c005210 0x7f240c00b9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f241400a730 tx=0x7f240c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38cbd8c8bb10930 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.488+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400a350 con 0x7f240c005210 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.488+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2414034650 con 0x7f240c005210 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.488+0000 7f2423fff640 1 -- 172.21.15.71:0/3895478633 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414034a40 con 0x7f240c005210 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.489+0000 7f242b7bd640 1 -- 172.21.15.71:0/3895478633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f240c005210 msgr2=0x7f240c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.489+0000 7f242b7bd640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f240c005210 0x7f240c00b9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f241400a730 tx=0x7f240c005830 comp rx=0 tx=0).stop 2024-04-16T15:41:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.489+0000 7f242b7bd640 1 -- 172.21.15.71:0/3895478633 shutdown_connections 2024-04-16T15:41:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.489+0000 7f242b7bd640 1 --2- 172.21.15.71:0/3895478633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f240c005210 0x7f240c00b9c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.489+0000 7f242b7bd640 1 -- 172.21.15.71:0/3895478633 >> 172.21.15.71:0/3895478633 conn(0x7f24240ab270 msgr2=0x7f24240ab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.489+0000 7f242b7bd640 1 -- 172.21.15.71:0/3895478633 shutdown_connections 2024-04-16T15:41:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.489+0000 7f242b7bd640 1 -- 172.21.15.71:0/3895478633 wait complete. 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.490+0000 7f242b7bd640 1 Processor -- start 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.490+0000 7f242b7bd640 1 -- start start 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f242b7bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c005210 0x7f24241eba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f242b7bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24241d9d50 0x7f242414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f242b7bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f242414e9b0 0x7f24241ee920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f242b7bd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f24240c5170 con 0x7f24241d9d50 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f242b7bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24240ac390 con 0x7f242414e9b0 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f242b7bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2424071630 con 0x7f240c005210 2024-04-16T15:41:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24241d9d50 0x7f242414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24241d9d50 0x7f242414e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58126/0 (socket says 172.21.15.71:58126) 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 -- 172.21.15.71:0/3322135768 learned_addr learned my addr 172.21.15.71:0/3322135768 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2429559640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c005210 0x7f24241eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2429d5a640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f242414e9b0 0x7f24241ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 -- 172.21.15.71:0/3322135768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c005210 msgr2=0x7f24241eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c005210 0x7f24241eba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 -- 172.21.15.71:0/3322135768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f242414e9b0 msgr2=0x7f24241ee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f242414e9b0 0x7f24241ee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 -- 172.21.15.71:0/3322135768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2424152110 con 0x7f24241d9d50 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2428d58640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24241d9d50 0x7f242414e470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2418011910 tx=0x7f2418011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2421ffb640 1 -- 172.21.15.71:0/3322135768 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241800faf0 con 0x7f24241d9d50 2024-04-16T15:41:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2421ffb640 1 -- 172.21.15.71:0/3322135768 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241800fc90 con 0x7f24241d9d50 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24241523f0 con 0x7f24241d9d50 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.491+0000 7f2421ffb640 1 -- 172.21.15.71:0/3322135768 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418014440 con 0x7f24241d9d50 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.492+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241529b0 con 0x7f24241d9d50 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.492+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2424068be0 con 0x7f24241d9d50 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.493+0000 7f2421ffb640 1 -- 172.21.15.71:0/3322135768 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2418012040 con 0x7f24241d9d50 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.493+0000 7f2421ffb640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f241006c920 0x7f241006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.493+0000 7f2421ffb640 1 -- 172.21.15.71:0/3322135768 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f241808bf20 con 0x7f24241d9d50 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.493+0000 7f2429559640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f241006c920 0x7f241006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.494+0000 7f2429559640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f241006c920 0x7f241006ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f241c000c00 tx=0x7f241c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.498+0000 7f2421ffb640 1 -- 172.21.15.71:0/3322135768 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2418019030 con 0x7f24241d9d50 2024-04-16T15:41:19.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.742+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f24240ad740 con 0x7f24241d9d50 2024-04-16T15:41:19.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.746+0000 7f2421ffb640 1 -- 172.21.15.71:0/3322135768 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f241805a5a0 con 0x7f24241d9d50 2024-04-16T15:41:19.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f241006c920 msgr2=0x7f241006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f241006c920 0x7f241006ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f241c000c00 tx=0x7f241c009040 comp rx=0 tx=0).stop 2024-04-16T15:41:19.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24241d9d50 msgr2=0x7f242414e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24241d9d50 0x7f242414e470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2418011910 tx=0x7f2418011dd0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 shutdown_connections 2024-04-16T15:41:19.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f241006c920 0x7f241006ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f242414e9b0 0x7f24241ee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24241d9d50 0x7f242414e470 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 --2- 172.21.15.71:0/3322135768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f240c005210 0x7f24241eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 >> 172.21.15.71:0/3322135768 conn(0x7f24240ab270 msgr2=0x7f24240ae550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:19.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 shutdown_connections 2024-04-16T15:41:19.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.748+0000 7f242b7bd640 1 -- 172.21.15.71:0/3322135768 wait complete. 2024-04-16T15:41:19.758 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-16T15:41:19.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.864+0000 7f3ae602c640 1 Processor -- start 2024-04-16T15:41:19.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.864+0000 7f3ae602c640 1 -- start start 2024-04-16T15:41:19.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ae602c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae014c9a0 0x7f3ae0153330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ae602c640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae0153870 con 0x7f3ae006e670 2024-04-16T15:41:19.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ae602c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae014cd80 con 0x7f3ae0156b70 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ae602c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae0155fd0 con 0x7f3ae014c9a0 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3adeffd640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f3ae006e670 0x7f3ae0156460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49724/0 (socket says 172.21.15.71:49724) 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3adeffd640 1 -- 172.21.15.71:0/3949441579 learned_addr learned my addr 172.21.15.71:0/3949441579 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617883703 0 0) 0x7f3ad4001110 con 0x7f3ae006e670 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ac8003520 con 0x7f3ae006e670 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3adf7fe640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae014c9a0 0x7f3ae0153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1886375171 0 0) 0x7f3ac8003520 con 0x7f3ae006e670 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae014c9a0 msgr2=0x7f3ae0153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae014c9a0 0x7f3ae0153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 >> v1:172.21.15.122:6789/0 conn(0x7f3ae0156b70 legacy=0x7f3ae015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae00c2f30 con 0x7f3ae006e670 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f3ad4001840 con 0x7f3ae006e670 2024-04-16T15:41:19.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.865+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 >> v1:172.21.15.71:6789/0 conn(0x7f3ae006e670 legacy=0x7f3ae0156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3ade7fc640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae014c9a0 0x7f3ac80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3ade7fc640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ac8003f60 0x7f3ac8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3ade7fc640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d40 0x7f3ac80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0155fd0 con 0x7f3ac8003f60 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac800bb50 con 0x7f3ac8004d40 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac800df60 con 0x7f3ae014c9a0 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3adeffd640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ac8003f60 0x7f3ac8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3adf7fe640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae014c9a0 0x7f3ac80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3adffff640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d40 0x7f3ac80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3adeffd640 1 -- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae014c9a0 msgr2=0x7f3ac80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3adeffd640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ae014c9a0 0x7f3ac80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3adeffd640 1 -- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d40 msgr2=0x7f3ac80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:19.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3adeffd640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d40 0x7f3ac80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.866+0000 7f3adeffd640 1 -- 172.21.15.71:0/3949441579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac8003520 con 0x7f3ac8003f60 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3adeffd640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ac8003f60 0x7f3ac8004800 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c4c0 tx=0x7f3ac8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c018f38c4bda6000 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad400c0e0 con 0x7f3ac8003f60 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4030070 con 0x7f3ac8003f60 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ade7fc640 1 -- 172.21.15.71:0/3949441579 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40058d0 con 0x7f3ac8003f60 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ae602c640 1 -- 172.21.15.71:0/3949441579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ac8003f60 msgr2=0x7f3ac8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ae602c640 1 --2- 172.21.15.71:0/3949441579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ac8003f60 0x7f3ac8004800 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c4c0 tx=0x7f3ac8005830 comp rx=0 tx=0).stop 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3adffff640 1 -- 172.21.15.71:0/3949441579 reap_dead start 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ae602c640 1 -- 172.21.15.71:0/3949441579 shutdown_connections 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ae602c640 1 -- 172.21.15.71:0/3949441579 >> 172.21.15.71:0/3949441579 conn(0x7f3ae006e220 msgr2=0x7f3ae00c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:19.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ae602c640 1 -- 172.21.15.71:0/3949441579 shutdown_connections 2024-04-16T15:41:19.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.867+0000 7f3ae602c640 1 -- 172.21.15.71:0/3949441579 wait complete. 2024-04-16T15:41:19.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.868+0000 7f3ae602c640 1 Processor -- start 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3ae602c640 1 -- start start 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3ae602c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ac8003f60 0x7f3ae01456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3ae602c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae0149cf0 0x7f3ae00733a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3ae602c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae00738e0 0x7f3ae014ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3ae602c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae00c3790 con 0x7f3ae0149cf0 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3ae602c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae00c34b0 con 0x7f3ae00738e0 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3ae602c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae00c3a70 con 0x7f3ac8003f60 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3adeffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae0149cf0 0x7f3ae00733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3adeffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae0149cf0 0x7f3ae00733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58152/0 (socket says 172.21.15.71:58152) 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.869+0000 7f3adeffd640 1 -- 172.21.15.71:0/1125152184 learned_addr learned my addr 172.21.15.71:0/1125152184 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3adeffd640 1 -- 172.21.15.71:0/1125152184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ac8003f60 msgr2=0x7f3ae01456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3adeffd640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ac8003f60 0x7f3ae01456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3adeffd640 1 -- 172.21.15.71:0/1125152184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae00738e0 msgr2=0x7f3ae014ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3adeffd640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae00738e0 0x7f3ae014ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3adeffd640 1 -- 172.21.15.71:0/1125152184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae01462f0 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3adeffd640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae0149cf0 0x7f3ae00733a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ad402c740 tx=0x7f3ad402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3ac7fff640 1 -- 172.21.15.71:0/1125152184 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007270 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3ac7fff640 1 -- 172.21.15.71:0/1125152184 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4007410 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae0146010 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3ac7fff640 1 -- 172.21.15.71:0/1125152184 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40077c0 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.870+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae01e8800 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.871+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae0068c10 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.871+0000 7f3ac7fff640 1 -- 172.21.15.71:0/1125152184 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ad4048020 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.872+0000 7f3ac7fff640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3acc06cac0 0x7f3acc06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.872+0000 7f3ac7fff640 1 -- 172.21.15.71:0/1125152184 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f3ad40aced0 con 0x7f3ae0149cf0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.872+0000 7f3adf7fe640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3acc06cac0 0x7f3acc06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:19.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.872+0000 7f3adf7fe640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3acc06cac0 0x7f3acc06ef80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009fc0 tx=0x7f3ad0009750 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:19.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:19.876+0000 7f3ac7fff640 1 -- 172.21.15.71:0/1125152184 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ae0153870 con 0x7f3ae0149cf0 2024-04-16T15:41:20.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.121+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f3ae00c4c40 con 0x7f3ae0149cf0 2024-04-16T15:41:20.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.125+0000 7f3ac7fff640 1 -- 172.21.15.71:0/1125152184 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f3ad407b550 con 0x7f3ae0149cf0 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3acc06cac0 msgr2=0x7f3acc06ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3acc06cac0 0x7f3acc06ef80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009fc0 tx=0x7f3ad0009750 comp rx=0 tx=0).stop 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae0149cf0 msgr2=0x7f3ae00733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae0149cf0 0x7f3ae00733a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ad402c740 tx=0x7f3ad402e070 comp rx=0 tx=0).stop 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 shutdown_connections 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3acc06cac0 0x7f3acc06ef80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae00738e0 0x7f3ae014ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae0149cf0 0x7f3ae00733a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 --2- 172.21.15.71:0/1125152184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3ac8003f60 0x7f3ae01456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 >> 172.21.15.71:0/1125152184 conn(0x7f3ae006e220 msgr2=0x7f3ae00aa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:20.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 shutdown_connections 2024-04-16T15:41:20.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.127+0000 7f3ae602c640 1 -- 172.21.15.71:0/1125152184 wait complete. 2024-04-16T15:41:20.137 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:20.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.240+0000 7f0b905bd640 1 Processor -- start 2024-04-16T15:41:20.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.241+0000 7f0b905bd640 1 -- start start 2024-04-16T15:41:20.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.241+0000 7f0b905bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 0x7f0b880aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.241+0000 7f0b905bd640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b88153ba0 con 0x7f0b88150f40 2024-04-16T15:41:20.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.241+0000 7f0b905bd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b88153900 con 0x7f0b880aa900 2024-04-16T15:41:20.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.241+0000 7f0b905bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0b880ad3e0 con 0x7f0b880ab950 2024-04-16T15:41:20.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8e359640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f0b88150f40 0x7f0b880accd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49740/0 (socket says 172.21.15.71:49740) 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8e359640 1 -- 172.21.15.71:0/580807436 learned_addr learned my addr 172.21.15.71:0/580807436 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8db58640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 0x7f0b880aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488540450 0 0) 0x7f0b88153ba0 con 0x7f0b88150f40 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b74003520 con 0x7f0b88150f40 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969689221 0 0) 0x7f0b88153900 con 0x7f0b880aa900 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b88153ba0 con 0x7f0b880aa900 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1398490090 0 0) 0x7f0b74003520 con 0x7f0b88150f40 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 msgr2=0x7f0b880aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 0x7f0b880aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 >> v1:172.21.15.122:6789/0 conn(0x7f0b880aa900 legacy=0x7f0b8815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b880ac4d0 con 0x7f0b88150f40 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f0b7c001670 con 0x7f0b88150f40 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8db58640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 0x7f0b880aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 >> v1:172.21.15.71:6789/0 conn(0x7f0b88150f40 legacy=0x7f0b880accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b74003b70 0x7f0b74003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 0x7f0b74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b74004d40 0x7f0b740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0b880ad3e0 con 0x7f0b74004d40 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7400baa0 con 0x7f0b74003b70 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7400bca0 con 0x7f0b880ab950 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8eb5a640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b74004d40 0x7f0b740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8db58640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 0x7f0b74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.242+0000 7f0b8e359640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b74003b70 0x7f0b74003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.243+0000 7f0b8eb5a640 1 -- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 msgr2=0x7f0b74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.243+0000 7f0b8eb5a640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b880ab950 0x7f0b74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.243+0000 7f0b8eb5a640 1 -- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b74003b70 msgr2=0x7f0b74003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.243+0000 7f0b8eb5a640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b74003b70 0x7f0b74003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.243+0000 7f0b8eb5a640 1 -- 172.21.15.71:0/580807436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b74003520 con 0x7f0b74004d40 2024-04-16T15:41:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.243+0000 7f0b8eb5a640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b74004d40 0x7f0b740051c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0b80001fd0 tx=0x7f0b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee61d6d3793a1a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.243+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80001bf0 con 0x7f0b74004d40 2024-04-16T15:41:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80003b70 con 0x7f0b74004d40 2024-04-16T15:41:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b8d357640 1 -- 172.21.15.71:0/580807436 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800050d0 con 0x7f0b74004d40 2024-04-16T15:41:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b905bd640 1 -- 172.21.15.71:0/580807436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b74004d40 msgr2=0x7f0b740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b905bd640 1 --2- 172.21.15.71:0/580807436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b74004d40 0x7f0b740051c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0b80001fd0 tx=0x7f0b74005830 comp rx=0 tx=0).stop 2024-04-16T15:41:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b8eb5a640 1 -- 172.21.15.71:0/580807436 reap_dead start 2024-04-16T15:41:20.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b905bd640 1 -- 172.21.15.71:0/580807436 shutdown_connections 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b905bd640 1 -- 172.21.15.71:0/580807436 >> 172.21.15.71:0/580807436 conn(0x7f0b880c3e10 msgr2=0x7f0b880c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b905bd640 1 -- 172.21.15.71:0/580807436 shutdown_connections 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.244+0000 7f0b905bd640 1 -- 172.21.15.71:0/580807436 wait complete. 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.245+0000 7f0b905bd640 1 Processor -- start 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.245+0000 7f0b905bd640 1 -- start start 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b905bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b74004d40 0x7f0b88151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b905bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b880c1230 0x7f0b881ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b905bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88149d10 0x7f0b88072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b905bd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8806fdd0 con 0x7f0b88149d10 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b905bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b880c1f20 con 0x7f0b880c1230 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b905bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8806ff50 con 0x7f0b74004d40 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8eb5a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88149d10 0x7f0b88072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8eb5a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88149d10 0x7f0b88072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58170/0 (socket says 172.21.15.71:58170) 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8eb5a640 1 -- 172.21.15.71:0/646050047 learned_addr learned my addr 172.21.15.71:0/646050047 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8e359640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b74004d40 0x7f0b88151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8db58640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b880c1230 0x7f0b881ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8eb5a640 1 -- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b74004d40 msgr2=0x7f0b88151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8eb5a640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b74004d40 0x7f0b88151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8eb5a640 1 -- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b880c1230 msgr2=0x7f0b881ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8eb5a640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b880c1230 0x7f0b881ecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8eb5a640 1 -- 172.21.15.71:0/646050047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b88151cf0 con 0x7f0b88149d10 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.246+0000 7f0b8db58640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b880c1230 0x7f0b881ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.247+0000 7f0b8eb5a640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88149d10 0x7f0b88072fb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b800020f0 tx=0x7f0b80001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.247+0000 7f0b7affd640 1 -- 172.21.15.71:0/646050047 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80042070 con 0x7f0b88149d10 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.247+0000 7f0b7affd640 1 -- 172.21.15.71:0/646050047 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80005380 con 0x7f0b88149d10 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.247+0000 7f0b7affd640 1 -- 172.21.15.71:0/646050047 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80039040 con 0x7f0b88149d10 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.247+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b880724d0 con 0x7f0b88149d10 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.247+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b88072a90 con 0x7f0b88149d10 2024-04-16T15:41:20.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.248+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b88072c90 con 0x7f0b88149d10 2024-04-16T15:41:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.248+0000 7f0b7affd640 1 -- 172.21.15.71:0/646050047 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b80045070 con 0x7f0b88149d10 2024-04-16T15:41:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.249+0000 7f0b7affd640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0b6c06c920 0x7f0b6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.249+0000 7f0b7affd640 1 -- 172.21.15.71:0/646050047 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f0b800ac220 con 0x7f0b88149d10 2024-04-16T15:41:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.249+0000 7f0b8e359640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0b6c06c920 0x7f0b6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.249+0000 7f0b8e359640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0b6c06c920 0x7f0b6c06ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c001050 tx=0x7f0b7c000f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.253+0000 7f0b7affd640 1 -- 172.21.15.71:0/646050047 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b88072c90 con 0x7f0b88149d10 2024-04-16T15:41:20.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.429+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b8806d680 con 0x7f0b88149d10 2024-04-16T15:41:20.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.430+0000 7f0b7affd640 1 -- 172.21.15.71:0/646050047 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13152 (secure 0 0 0) 0x7f0b8007a250 con 0x7f0b88149d10 2024-04-16T15:41:20.429 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:20.429 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":34,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:17.486122+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0b6c06c920 msgr2=0x7f0b6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0b6c06c920 0x7f0b6c06ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c001050 tx=0x7f0b7c000f80 comp rx=0 tx=0).stop 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88149d10 msgr2=0x7f0b88072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88149d10 0x7f0b88072fb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b800020f0 tx=0x7f0b80001650 comp rx=0 tx=0).stop 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 shutdown_connections 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f0b6c06c920 0x7f0b6c06ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b88149d10 0x7f0b88072fb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b880c1230 0x7f0b881ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 --2- 172.21.15.71:0/646050047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b74004d40 0x7f0b88151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 >> 172.21.15.71:0/646050047 conn(0x7f0b880c3e10 msgr2=0x7f0b88062440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:20.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 shutdown_connections 2024-04-16T15:41:20.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.432+0000 7f0b905bd640 1 -- 172.21.15.71:0/646050047 wait complete. 2024-04-16T15:41:20.442 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T15:40:11.711384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T15:41:20.442 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T15:41:20.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.553+0000 7fe4e1e4e640 1 Processor -- start 2024-04-16T15:41:20.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.553+0000 7fe4e1e4e640 1 -- start start 2024-04-16T15:41:20.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4e1e4e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe4dc0c8c10 0x7fe4dc0c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4e1e4e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4dc155d30 con 0x7fe4dc1564f0 2024-04-16T15:41:20.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4e1e4e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4dc0c3d50 con 0x7fe4dc153320 2024-04-16T15:41:20.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4e1e4e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe4dc0af4c0 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4dbfff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fe4dc1564f0 0x7fe4dc071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49754/0 (socket says 172.21.15.71:49754) 2024-04-16T15:41:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4dbfff640 1 -- 172.21.15.71:0/2785061406 learned_addr learned my addr 172.21.15.71:0/2785061406 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4daffd640 1 --2- 172.21.15.71:0/2785061406 >> v2:172.21.15.122:3300/0 conn(0x7fe4dc0c8c10 0x7fe4dc0c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4da7fc640 1 -- 172.21.15.71:0/2785061406 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3247456484 0 0) 0x7fe4dc155d30 con 0x7fe4dc1564f0 2024-04-16T15:41:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4da7fc640 1 -- 172.21.15.71:0/2785061406 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4c4003520 con 0x7fe4dc1564f0 2024-04-16T15:41:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4da7fc640 1 -- 172.21.15.71:0/2785061406 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1587133655 0 0) 0x7fe4dc0c3d50 con 0x7fe4dc153320 2024-04-16T15:41:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.554+0000 7fe4da7fc640 1 -- 172.21.15.71:0/2785061406 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4dc155d30 con 0x7fe4dc153320 2024-04-16T15:41:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.555+0000 7fe4daffd640 1 -- 172.21.15.71:0/2785061406 >> v1:172.21.15.122:6789/0 conn(0x7fe4dc153320 legacy=0x7fe4dc0c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.555+0000 7fe4daffd640 1 -- 172.21.15.71:0/2785061406 >> v1:172.21.15.71:6789/0 conn(0x7fe4dc1564f0 legacy=0x7fe4dc071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.555+0000 7fe4da7fc640 1 -- 172.21.15.71:0/2785061406 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3941247885 0 0) 0x7fe4c4003520 con 0x7fe4dc1564f0 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.555+0000 7fe4daffd640 1 -- 172.21.15.71:0/2785061406 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc0c4030 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.555+0000 7fe4daffd640 1 --2- 172.21.15.71:0/2785061406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc0c26e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00097a0 tx=0x7fe4d002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbb1686b49db34d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.555+0000 7fe4da7fc640 1 -- 172.21.15.71:0/2785061406 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0037070 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4da7fc640 1 -- 172.21.15.71:0/2785061406 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d0030510 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4da7fc640 1 -- 172.21.15.71:0/2785061406 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4001a10 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2785061406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc0c8c10 msgr2=0x7fe4dc0c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4e1e4e640 1 --2- 172.21.15.71:0/2785061406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc0c26e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00097a0 tx=0x7fe4d002b710 comp rx=0 tx=0).stop 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2785061406 shutdown_connections 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4e1e4e640 1 --2- 172.21.15.71:0/2785061406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc0c26e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2785061406 >> 172.21.15.71:0/2785061406 conn(0x7fe4dc0c5db0 msgr2=0x7fe4dc0c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:20.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2785061406 shutdown_connections 2024-04-16T15:41:20.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.556+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2785061406 wait complete. 2024-04-16T15:41:20.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.557+0000 7fe4e1e4e640 1 Processor -- start 2024-04-16T15:41:20.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.557+0000 7fe4e1e4e640 1 -- start start 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4e1e4e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc14ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4e1e4e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4dc153320 0x7fe4dc1e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4e1e4e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc1564f0 0x7fe4dc1ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4e1e4e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc072450 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4e1e4e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc0722d0 con 0x7fe4dc1564f0 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4e1e4e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc156940 con 0x7fe4dc153320 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc14ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc14ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58182/0 (socket says 172.21.15.71:58182) 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4db7fe640 1 -- 172.21.15.71:0/2646424285 learned_addr learned my addr 172.21.15.71:0/2646424285 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4dbfff640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc1564f0 0x7fe4dc1ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4daffd640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4dc153320 0x7fe4dc1e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4db7fe640 1 -- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4dc153320 msgr2=0x7fe4dc1e1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4db7fe640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4dc153320 0x7fe4dc1e1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4db7fe640 1 -- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc1564f0 msgr2=0x7fe4dc1ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4db7fe640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc1564f0 0x7fe4dc1ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4db7fe640 1 -- 172.21.15.71:0/2646424285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc14d060 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.558+0000 7fe4dbfff640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc1564f0 0x7fe4dc1ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.559+0000 7fe4db7fe640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc14ea00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c500 tx=0x7fe4cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.559+0000 7fe4c3fff640 1 -- 172.21.15.71:0/2646424285 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0052d0 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.559+0000 7fe4c3fff640 1 -- 172.21.15.71:0/2646424285 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc005490 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.559+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc14d340 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.559+0000 7fe4c3fff640 1 -- 172.21.15.71:0/2646424285 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc012870 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.559+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc159e20 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.560+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc069510 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.560+0000 7fe4c3fff640 1 -- 172.21.15.71:0/2646424285 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4cc012a10 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.560+0000 7fe4c3fff640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe4c806c920 0x7fe4c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.561+0000 7fe4c3fff640 1 -- 172.21.15.71:0/2646424285 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fe4cc08b630 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.561+0000 7fe4daffd640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe4c806c920 0x7fe4c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.561+0000 7fe4daffd640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe4c806c920 0x7fe4c806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0034000 tx=0x7fe4d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.565+0000 7fe4c3fff640 1 -- 172.21.15.71:0/2646424285 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4dc069510 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.740+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4dc14d510 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.740+0000 7fe4c3fff640 1 -- 172.21.15.71:0/2646424285 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fe4cc059c30 con 0x7fe4dc0c8c10 2024-04-16T15:41:20.739 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-16T15:41:20.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe4c806c920 msgr2=0x7fe4c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe4c806c920 0x7fe4c806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0034000 tx=0x7fe4d0008030 comp rx=0 tx=0).stop 2024-04-16T15:41:20.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4dc0c8c10 msgr2=0x7fe4dc14ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc14ea00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c500 tx=0x7fe4cc00c9c0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 shutdown_connections 2024-04-16T15:41:20.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fe4c806c920 0x7fe4c806ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4dc1564f0 0x7fe4dc1ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4dc153320 0x7fe4dc1e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 --2- 172.21.15.71:0/2646424285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4dc0c8c10 0x7fe4dc14ea00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 >> 172.21.15.71:0/2646424285 conn(0x7fe4dc0c5db0 msgr2=0x7fe4dc154670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:20.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 shutdown_connections 2024-04-16T15:41:20.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.742+0000 7fe4e1e4e640 1 -- 172.21.15.71:0/2646424285 wait complete. 2024-04-16T15:41:20.752 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-16T15:41:20.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.856+0000 7f20d38d9640 1 Processor -- start 2024-04-16T15:41:20.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.856+0000 7f20d38d9640 1 -- start start 2024-04-16T15:41:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d38d9640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f20cc06fac0 0x7f20cc0c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d38d9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc0ab290 0x7f20cc15d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d38d9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20cc0c27c0 con 0x7f20cc155dc0 2024-04-16T15:41:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d38d9640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f20cc073f20 con 0x7f20cc06fac0 2024-04-16T15:41:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d38d9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f20cc156ab0 con 0x7f20cc0ab290 2024-04-16T15:41:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d0e74640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f20cc06fac0 0x7f20cc0c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d0e74640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f20cc06fac0 0x7f20cc0c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58196/0 (socket says 172.21.15.71:58196) 2024-04-16T15:41:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d0e74640 1 -- 172.21.15.71:0/745976714 learned_addr learned my addr 172.21.15.71:0/745976714 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.857+0000 7f20d1e76640 1 --2- 172.21.15.71:0/745976714 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc0ab290 0x7f20cc15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.858+0000 7f20d0e74640 1 -- 172.21.15.71:0/745976714 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc0ab290 msgr2=0x7f20cc15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.858+0000 7f20d0e74640 1 --2- 172.21.15.71:0/745976714 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc0ab290 0x7f20cc15d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.858+0000 7f20d0e74640 1 -- 172.21.15.71:0/745976714 >> v1:172.21.15.122:6789/0 conn(0x7f20cc155dc0 legacy=0x7f20cc1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.858+0000 7f20d0e74640 1 -- 172.21.15.71:0/745976714 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc0c2aa0 con 0x7f20cc06fac0 2024-04-16T15:41:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.858+0000 7f20d0e74640 1 --2- 172.21.15.71:0/745976714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc06fac0 0x7f20cc0c32a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009870 tx=0x7f20c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d88f4d6bee6dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.858+0000 7f20cbfff640 1 -- 172.21.15.71:0/745976714 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0034070 con 0x7f20cc06fac0 2024-04-16T15:41:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.858+0000 7f20cbfff640 1 -- 172.21.15.71:0/745976714 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c0030840 con 0x7f20cc06fac0 2024-04-16T15:41:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.858+0000 7f20cbfff640 1 -- 172.21.15.71:0/745976714 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0030c10 con 0x7f20cc06fac0 2024-04-16T15:41:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.859+0000 7f20d38d9640 1 -- 172.21.15.71:0/745976714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc06fac0 msgr2=0x7f20cc0c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.859+0000 7f20d38d9640 1 --2- 172.21.15.71:0/745976714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc06fac0 0x7f20cc0c32a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009870 tx=0x7f20c002b980 comp rx=0 tx=0).stop 2024-04-16T15:41:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.859+0000 7f20d38d9640 1 -- 172.21.15.71:0/745976714 shutdown_connections 2024-04-16T15:41:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.859+0000 7f20d38d9640 1 --2- 172.21.15.71:0/745976714 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc0ab290 0x7f20cc15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.859+0000 7f20d38d9640 1 --2- 172.21.15.71:0/745976714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc06fac0 0x7f20cc0c32a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.859+0000 7f20d38d9640 1 -- 172.21.15.71:0/745976714 >> 172.21.15.71:0/745976714 conn(0x7f20cc0c3e10 msgr2=0x7f20cc0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.859+0000 7f20d38d9640 1 -- 172.21.15.71:0/745976714 shutdown_connections 2024-04-16T15:41:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.859+0000 7f20d38d9640 1 -- 172.21.15.71:0/745976714 wait complete. 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.860+0000 7f20d38d9640 1 Processor -- start 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.860+0000 7f20d38d9640 1 -- start start 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d38d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc0ab290 0x7f20cc1ed150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d38d9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc155dc0 0x7f20cc1edae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d38d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20cc1464d0 0x7f20cc1ea340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d38d9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc06bd70 con 0x7f20cc0ab290 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d38d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0c18b0 con 0x7f20cc1464d0 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d38d9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f20cc06bef0 con 0x7f20cc155dc0 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc0ab290 0x7f20cc1ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc0ab290 0x7f20cc1ed150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58208/0 (socket says 172.21.15.71:58208) 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 -- 172.21.15.71:0/1618102402 learned_addr learned my addr 172.21.15.71:0/1618102402 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1e76640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20cc1464d0 0x7f20cc1ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d0e74640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc155dc0 0x7f20cc1edae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 -- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc155dc0 msgr2=0x7f20cc1edae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc155dc0 0x7f20cc1edae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 -- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20cc1464d0 msgr2=0x7f20cc1ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20cc1464d0 0x7f20cc1ea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 -- 172.21.15.71:0/1618102402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc1e9b60 con 0x7f20cc0ab290 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d0e74640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc155dc0 0x7f20cc1edae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.861+0000 7f20d1675640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc0ab290 0x7f20cc1ed150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00c3c0 tx=0x7f20bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.862+0000 7f20c9ffb640 1 -- 172.21.15.71:0/1618102402 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0052d0 con 0x7f20cc0ab290 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.862+0000 7f20c9ffb640 1 -- 172.21.15.71:0/1618102402 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20bc005490 con 0x7f20cc0ab290 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.862+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc0c6430 con 0x7f20cc0ab290 2024-04-16T15:41:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.862+0000 7f20c9ffb640 1 -- 172.21.15.71:0/1618102402 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc007260 con 0x7f20cc0ab290 2024-04-16T15:41:20.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.862+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc0c69f0 con 0x7f20cc0ab290 2024-04-16T15:41:20.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.863+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20cc149420 con 0x7f20cc0ab290 2024-04-16T15:41:20.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.863+0000 7f20c9ffb640 1 -- 172.21.15.71:0/1618102402 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20bc007400 con 0x7f20cc0ab290 2024-04-16T15:41:20.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.864+0000 7f20c9ffb640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20b806c8f0 0x7f20b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:20.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.864+0000 7f20c9ffb640 1 -- 172.21.15.71:0/1618102402 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f20bc08ae00 con 0x7f20cc0ab290 2024-04-16T15:41:20.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.864+0000 7f20d0e74640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20b806c8f0 0x7f20b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:20.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.864+0000 7f20d0e74640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20b806c8f0 0x7f20b806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20c0036000 tx=0x7f20c003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:20.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:20.868+0000 7f20c9ffb640 1 -- 172.21.15.71:0/1618102402 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20cc149420 con 0x7f20cc0ab290 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.046+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f20cc14d2b0 con 0x7f20cc0ab290 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.046+0000 7f20c9ffb640 1 -- 172.21.15.71:0/1618102402 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f20bc059480 con 0x7f20cc0ab290 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-16T15:41:21.045 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-04-16T15:41:21.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20b806c8f0 msgr2=0x7f20b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20b806c8f0 0x7f20b806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20c0036000 tx=0x7f20c003b040 comp rx=0 tx=0).stop 2024-04-16T15:41:21.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc0ab290 msgr2=0x7f20cc1ed150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc0ab290 0x7f20cc1ed150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00c3c0 tx=0x7f20bc00c880 comp rx=0 tx=0).stop 2024-04-16T15:41:21.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 shutdown_connections 2024-04-16T15:41:21.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f20b806c8f0 0x7f20b806edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20cc1464d0 0x7f20cc1ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20cc155dc0 0x7f20cc1edae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 --2- 172.21.15.71:0/1618102402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20cc0ab290 0x7f20cc1ed150 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 >> 172.21.15.71:0/1618102402 conn(0x7f20cc0c3e10 msgr2=0x7f20cc13eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:21.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 shutdown_connections 2024-04-16T15:41:21.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.048+0000 7f20d38d9640 1 -- 172.21.15.71:0/1618102402 wait complete. 2024-04-16T15:41:21.058 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-16T15:41:21.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-16T15:41:21.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.166+0000 7f2ac6493640 1 Processor -- start 2024-04-16T15:41:21.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.166+0000 7f2ac6493640 1 -- start start 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.167+0000 7f2ac6493640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 0x7f2ac006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.167+0000 7f2ac6493640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac00acbf0 con 0x7f2ac0156050 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.167+0000 7f2ac6493640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac00aceb0 con 0x7f2ac006cf50 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.167+0000 7f2ac6493640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac014fbb0 con 0x7f2ac00ac510 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.167+0000 7f2ac4a30640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f2ac0156050 0x7f2ac00722b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49766/0 (socket says 172.21.15.71:49766) 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.167+0000 7f2ac4a30640 1 -- 172.21.15.71:0/1223095919 learned_addr learned my addr 172.21.15.71:0/1223095919 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.167+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989357950 0 0) 0x7f2ac00acbf0 con 0x7f2ac0156050 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abffff640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 0x7f2ac006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aa8003440 con 0x7f2ac0156050 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3248437469 0 0) 0x7f2aa8003440 con 0x7f2ac0156050 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 msgr2=0x7f2ac006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 0x7f2ac006c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 >> v1:172.21.15.122:6789/0 conn(0x7f2ac006cf50 legacy=0x7f2ac0071ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac00ad190 con 0x7f2ac0156050 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f2ab80019f0 con 0x7f2ac0156050 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abffff640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 0x7f2ac006c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 >> v1:172.21.15.71:6789/0 conn(0x7f2ac0156050 legacy=0x7f2ac00722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa8003ad0 0x7f2aa8003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa8008970 0x7f2aa800ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 0x7f2aa800d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac014fbb0 con 0x7f2aa8003ad0 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa800de10 con 0x7f2aa8008970 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa800e010 con 0x7f2ac00ac510 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abffff640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa8003ad0 0x7f2aa8003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abf7fe640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa8008970 0x7f2aa800ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2ac4a30640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 0x7f2aa800d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abffff640 1 -- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 msgr2=0x7f2aa800d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abffff640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2ac00ac510 0x7f2aa800d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abffff640 1 -- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa8008970 msgr2=0x7f2aa800ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abffff640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa8008970 0x7f2aa800ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.168+0000 7f2abffff640 1 -- 172.21.15.71:0/1223095919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa8003440 con 0x7f2aa8003ad0 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.169+0000 7f2abffff640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa8003ad0 0x7f2aa8003ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00098a0 tx=0x7f2aa8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5ad52dad5b9bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.169+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab000f040 con 0x7f2aa8003ad0 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.169+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab0004230 con 0x7f2aa8003ad0 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.169+0000 7f2abeffd640 1 -- 172.21.15.71:0/1223095919 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0003610 con 0x7f2aa8003ad0 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.170+0000 7f2ac6493640 1 -- 172.21.15.71:0/1223095919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa8003ad0 msgr2=0x7f2aa8003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.170+0000 7f2ac6493640 1 --2- 172.21.15.71:0/1223095919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2aa8003ad0 0x7f2aa8003ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00098a0 tx=0x7f2aa8003730 comp rx=0 tx=0).stop 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.170+0000 7f2ac4a30640 1 -- 172.21.15.71:0/1223095919 reap_dead start 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.170+0000 7f2ac6493640 1 -- 172.21.15.71:0/1223095919 shutdown_connections 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.170+0000 7f2ac6493640 1 -- 172.21.15.71:0/1223095919 >> 172.21.15.71:0/1223095919 conn(0x7f2ac00c3e80 msgr2=0x7f2ac00c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.170+0000 7f2ac6493640 1 -- 172.21.15.71:0/1223095919 shutdown_connections 2024-04-16T15:41:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.170+0000 7f2ac6493640 1 -- 172.21.15.71:0/1223095919 wait complete. 2024-04-16T15:41:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.171+0000 7f2ac6493640 1 Processor -- start 2024-04-16T15:41:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.171+0000 7f2ac6493640 1 -- start start 2024-04-16T15:41:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac6493640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2aa8003ad0 0x7f2ac01efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac6493640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac01500f0 0x7f2ac00c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac6493640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ac0150f00 0x7f2ac0151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac6493640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac00c3b00 con 0x7f2ac0150f00 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac6493640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac00c3980 con 0x7f2ac01500f0 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac6493640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac00c3c80 con 0x7f2aa8003ad0 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ac0150f00 0x7f2ac0151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ac0150f00 0x7f2ac0151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58222/0 (socket says 172.21.15.71:58222) 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 -- 172.21.15.71:0/3044688004 learned_addr learned my addr 172.21.15.71:0/3044688004 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2abffff640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2aa8003ad0 0x7f2ac01efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2abf7fe640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac01500f0 0x7f2ac00c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 -- 172.21.15.71:0/3044688004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2aa8003ad0 msgr2=0x7f2ac01efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2aa8003ad0 0x7f2ac01efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 -- 172.21.15.71:0/3044688004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac01500f0 msgr2=0x7f2ac00c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac01500f0 0x7f2ac00c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 -- 172.21.15.71:0/3044688004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac01518c0 con 0x7f2ac0150f00 2024-04-16T15:41:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2ac4a30640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ac0150f00 0x7f2ac0151380 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8000f80 tx=0x7f2ab8007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.172+0000 7f2abcff9640 1 -- 172.21.15.71:0/3044688004 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8017020 con 0x7f2ac0150f00 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.173+0000 7f2abcff9640 1 -- 172.21.15.71:0/3044688004 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab800ab00 con 0x7f2ac0150f00 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.173+0000 7f2abcff9640 1 -- 172.21.15.71:0/3044688004 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80133f0 con 0x7f2ac0150f00 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.173+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac01f0240 con 0x7f2ac0150f00 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.173+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01ec0e0 con 0x7f2ac0150f00 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.173+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac0068be0 con 0x7f2ac0150f00 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.174+0000 7f2abcff9640 1 -- 172.21.15.71:0/3044688004 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ab801a020 con 0x7f2ac0150f00 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.174+0000 7f2abcff9640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.174+0000 7f2abcff9640 1 -- 172.21.15.71:0/3044688004 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f2ab808bf10 con 0x7f2ac0150f00 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.175+0000 7f2abffff640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.175+0000 7f2abffff640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2aac06c8f0 0x7f2aac06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0035000 tx=0x7f2ab0031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:21.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.179+0000 7f2abcff9640 1 -- 172.21.15.71:0/3044688004 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ab8021020 con 0x7f2ac0150f00 2024-04-16T15:41:21.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.355+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2ac006cca0 con 0x7f2ac0150f00 2024-04-16T15:41:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.497+0000 7f2abcff9640 1 -- 172.21.15.71:0/3044688004 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f2ac01518c0 con 0x7f2ac0150f00 2024-04-16T15:41:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-04-16T15:41:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.499+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2aac06c8f0 msgr2=0x7f2aac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.499+0000 7f2ac6493640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2aac06c8f0 0x7f2aac06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0035000 tx=0x7f2ab0031000 comp rx=0 tx=0).stop 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.499+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ac0150f00 msgr2=0x7f2ac0151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.499+0000 7f2ac6493640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ac0150f00 0x7f2ac0151380 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8000f80 tx=0x7f2ab8007b20 comp rx=0 tx=0).stop 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.500+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 shutdown_connections 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.500+0000 7f2ac6493640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f2aac06c8f0 0x7f2aac06edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.500+0000 7f2ac6493640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ac0150f00 0x7f2ac0151380 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.500+0000 7f2ac6493640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ac01500f0 0x7f2ac00c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.500+0000 7f2ac6493640 1 --2- 172.21.15.71:0/3044688004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2aa8003ad0 0x7f2ac01efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.500+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 >> 172.21.15.71:0/3044688004 conn(0x7f2ac00c3e80 msgr2=0x7f2ac00aabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.500+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 shutdown_connections 2024-04-16T15:41:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.500+0000 7f2ac6493640 1 -- 172.21.15.71:0/3044688004 wait complete. 2024-04-16T15:41:21.512 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-16T15:41:21.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.623+0000 7f0372435640 1 Processor -- start 2024-04-16T15:41:21.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.623+0000 7f0372435640 1 -- start start 2024-04-16T15:41:21.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f0372435640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f036c072240 0x7f036c160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f0372435640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036c0c5b00 con 0x7f036c06cbb0 2024-04-16T15:41:21.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f0372435640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036c0c6390 con 0x7f036c0c3f40 2024-04-16T15:41:21.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f0372435640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f036c0c6670 con 0x7f036c072240 2024-04-16T15:41:21.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f03709d2640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f036c072240 0x7f036c160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f036bfff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f036c06cbb0 0x7f036c0c2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49780/0 (socket says 172.21.15.71:49780) 2024-04-16T15:41:21.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f036bfff640 1 -- 172.21.15.71:0/165464145 learned_addr learned my addr 172.21.15.71:0/165464145 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:21.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f03709d2640 1 -- 172.21.15.71:0/165464145 >> v1:172.21.15.122:6789/0 conn(0x7f036c0c3f40 legacy=0x7f036c0c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:21.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f03709d2640 1 -- 172.21.15.71:0/165464145 >> v1:172.21.15.71:6789/0 conn(0x7f036c06cbb0 legacy=0x7f036c0c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f03709d2640 1 -- 172.21.15.71:0/165464145 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c0c5da0 con 0x7f036c072240 2024-04-16T15:41:21.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.624+0000 7f036affd640 1 -- 172.21.15.71:0/165464145 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650702826 0 0) 0x7f036c0c5b00 con 0x7f036c06cbb0 2024-04-16T15:41:21.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.625+0000 7f03709d2640 1 --2- 172.21.15.71:0/165464145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c072240 0x7f036c160990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0364009d90 tx=0x7f036402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05dcd649b3b225c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:21.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.625+0000 7f036affd640 1 -- 172.21.15.71:0/165464145 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364039070 con 0x7f036c072240 2024-04-16T15:41:21.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.625+0000 7f036affd640 1 -- 172.21.15.71:0/165464145 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f036402cb40 con 0x7f036c072240 2024-04-16T15:41:21.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.625+0000 7f036affd640 1 -- 172.21.15.71:0/165464145 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364034440 con 0x7f036c072240 2024-04-16T15:41:21.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.626+0000 7f0372435640 1 -- 172.21.15.71:0/165464145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c072240 msgr2=0x7f036c160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.626+0000 7f0372435640 1 --2- 172.21.15.71:0/165464145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c072240 0x7f036c160990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0364009d90 tx=0x7f036402bd90 comp rx=0 tx=0).stop 2024-04-16T15:41:21.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.626+0000 7f0372435640 1 -- 172.21.15.71:0/165464145 shutdown_connections 2024-04-16T15:41:21.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.626+0000 7f0372435640 1 --2- 172.21.15.71:0/165464145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c072240 0x7f036c160990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.626+0000 7f0372435640 1 -- 172.21.15.71:0/165464145 >> 172.21.15.71:0/165464145 conn(0x7f036c0c45d0 msgr2=0x7f036c0c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:21.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.626+0000 7f0372435640 1 -- 172.21.15.71:0/165464145 shutdown_connections 2024-04-16T15:41:21.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.626+0000 7f0372435640 1 -- 172.21.15.71:0/165464145 wait complete. 2024-04-16T15:41:21.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.627+0000 7f0372435640 1 Processor -- start 2024-04-16T15:41:21.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.627+0000 7f0372435640 1 -- start start 2024-04-16T15:41:21.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f0372435640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c06cbb0 0x7f036c1e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f0372435640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f036c072240 0x7f036c1d4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f0372435640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f036c0c3f40 0x7f036c1d5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f0372435640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f036c06cf90 con 0x7f036c06cbb0 2024-04-16T15:41:21.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f0372435640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f036c073060 con 0x7f036c072240 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f0372435640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f036c06d110 con 0x7f036c0c3f40 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c06cbb0 0x7f036c1e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c06cbb0 0x7f036c1e0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58230/0 (socket says 172.21.15.71:58230) 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036bfff640 1 -- 172.21.15.71:0/2181792255 learned_addr learned my addr 172.21.15.71:0/2181792255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f03709d2640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f036c0c3f40 0x7f036c1d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036b7fe640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f036c072240 0x7f036c1d4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036bfff640 1 -- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f036c0c3f40 msgr2=0x7f036c1d5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036bfff640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f036c0c3f40 0x7f036c1d5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036bfff640 1 -- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f036c072240 msgr2=0x7f036c1d4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036bfff640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f036c072240 0x7f036c1d4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036bfff640 1 -- 172.21.15.71:0/2181792255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c1def40 con 0x7f036c06cbb0 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.628+0000 7f036b7fe640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f036c072240 0x7f036c1d4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.629+0000 7f036bfff640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c06cbb0 0x7f036c1e0c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f035c00c4d0 tx=0x7f035c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.629+0000 7f0368ff9640 1 -- 172.21.15.71:0/2181792255 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c018070 con 0x7f036c06cbb0 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.629+0000 7f0368ff9640 1 -- 172.21.15.71:0/2181792255 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035c007500 con 0x7f036c06cbb0 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.629+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036c1d6ed0 con 0x7f036c06cbb0 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.629+0000 7f0368ff9640 1 -- 172.21.15.71:0/2181792255 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c004a20 con 0x7f036c06cbb0 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.629+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036c1dec60 con 0x7f036c06cbb0 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.630+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036c068d20 con 0x7f036c06cbb0 2024-04-16T15:41:21.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.630+0000 7f0368ff9640 1 -- 172.21.15.71:0/2181792255 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f035c01b070 con 0x7f036c06cbb0 2024-04-16T15:41:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.631+0000 7f0368ff9640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f035806c8f0 0x7f035806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.631+0000 7f0368ff9640 1 -- 172.21.15.71:0/2181792255 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f035c08ad70 con 0x7f036c06cbb0 2024-04-16T15:41:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.631+0000 7f036b7fe640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f035806c8f0 0x7f035806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.631+0000 7f036b7fe640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f035806c8f0 0x7f035806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f036000a830 tx=0x7f0360008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:21.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.636+0000 7f0368ff9640 1 -- 172.21.15.71:0/2181792255 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f036c1def40 con 0x7f036c06cbb0 2024-04-16T15:41:21.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:21.812+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f036c0a8250 con 0x7f036c06cbb0 2024-04-16T15:41:22.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.503+0000 7f0368ff9640 1 -- 172.21.15.71:0/2181792255 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f035c059280 con 0x7f036c06cbb0 2024-04-16T15:41:22.502 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-04-16T15:41:22.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f035806c8f0 msgr2=0x7f035806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:22.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f035806c8f0 0x7f035806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f036000a830 tx=0x7f0360008040 comp rx=0 tx=0).stop 2024-04-16T15:41:22.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c06cbb0 msgr2=0x7f036c1e0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:22.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c06cbb0 0x7f036c1e0c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f035c00c4d0 tx=0x7f035c00c990 comp rx=0 tx=0).stop 2024-04-16T15:41:22.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 shutdown_connections 2024-04-16T15:41:22.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f035806c8f0 0x7f035806edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f036c0c3f40 0x7f036c1d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f036c072240 0x7f036c1d4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 --2- 172.21.15.71:0/2181792255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f036c06cbb0 0x7f036c1e0c50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 >> 172.21.15.71:0/2181792255 conn(0x7f036c0c45d0 msgr2=0x7f036c0a8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:22.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 shutdown_connections 2024-04-16T15:41:22.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.506+0000 7f0372435640 1 -- 172.21.15.71:0/2181792255 wait complete. 2024-04-16T15:41:22.531 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-16T15:41:22.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.642+0000 7f2498ad3640 1 Processor -- start 2024-04-16T15:41:22.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.642+0000 7f2498ad3640 1 -- start start 2024-04-16T15:41:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2498ad3640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f24940acc00 0x7f24940ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2498ad3640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2494152340 0x7f24941527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2498ad3640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24940c3700 con 0x7f249414f840 2024-04-16T15:41:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2498ad3640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f24940c2780 con 0x7f2494152340 2024-04-16T15:41:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2498ad3640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f24940c2cb0 con 0x7f24940acc00 2024-04-16T15:41:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2492d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2494152340 0x7f24941527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:22.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2492d9d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2494152340 0x7f24941527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58240/0 (socket says 172.21.15.71:58240) 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2492d9d640 1 -- 172.21.15.71:0/1249390833 learned_addr learned my addr 172.21.15.71:0/1249390833 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.643+0000 7f2491d9b640 1 --2- 172.21.15.71:0/1249390833 >> v2:172.21.15.122:3300/0 conn(0x7f24940acc00 0x7f24940ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.644+0000 7f2492d9d640 1 -- 172.21.15.71:0/1249390833 >> v2:172.21.15.122:3300/0 conn(0x7f24940acc00 msgr2=0x7f24940ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.644+0000 7f2492d9d640 1 --2- 172.21.15.71:0/1249390833 >> v2:172.21.15.122:3300/0 conn(0x7f24940acc00 0x7f24940ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.644+0000 7f2492d9d640 1 -- 172.21.15.71:0/1249390833 >> v1:172.21.15.122:6789/0 conn(0x7f249414f840 legacy=0x7f24940719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.644+0000 7f2492d9d640 1 -- 172.21.15.71:0/1249390833 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24940c39a0 con 0x7f2494152340 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.644+0000 7f2491d9b640 1 --2- 172.21.15.71:0/1249390833 >> v2:172.21.15.122:3300/0 conn(0x7f24940acc00 0x7f24940ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.644+0000 7f2492d9d640 1 --2- 172.21.15.71:0/1249390833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2494152340 0x7f24941527f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f24940716a0 tx=0x7f248c02de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80462fe4fef50f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.644+0000 7f249159a640 1 -- 172.21.15.71:0/1249390833 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c034070 con 0x7f2494152340 2024-04-16T15:41:22.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f249159a640 1 -- 172.21.15.71:0/1249390833 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248c02ebd0 con 0x7f2494152340 2024-04-16T15:41:22.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f249159a640 1 -- 172.21.15.71:0/1249390833 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c038400 con 0x7f2494152340 2024-04-16T15:41:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f2498ad3640 1 -- 172.21.15.71:0/1249390833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2494152340 msgr2=0x7f24941527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f2498ad3640 1 --2- 172.21.15.71:0/1249390833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2494152340 0x7f24941527f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f24940716a0 tx=0x7f248c02de20 comp rx=0 tx=0).stop 2024-04-16T15:41:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f2498ad3640 1 -- 172.21.15.71:0/1249390833 shutdown_connections 2024-04-16T15:41:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f2498ad3640 1 --2- 172.21.15.71:0/1249390833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2494152340 0x7f24941527f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f2498ad3640 1 --2- 172.21.15.71:0/1249390833 >> v2:172.21.15.122:3300/0 conn(0x7f24940acc00 0x7f24940ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f2498ad3640 1 -- 172.21.15.71:0/1249390833 >> 172.21.15.71:0/1249390833 conn(0x7f24940c7940 msgr2=0x7f24940c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f2498ad3640 1 -- 172.21.15.71:0/1249390833 shutdown_connections 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.645+0000 7f2498ad3640 1 -- 172.21.15.71:0/1249390833 wait complete. 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.646+0000 7f2498ad3640 1 Processor -- start 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2498ad3640 1 -- start start 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2498ad3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24940acc00 0x7f2494147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2498ad3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249414f840 0x7f24940c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2498ad3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2494152340 0x7f24941eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2498ad3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2494072240 con 0x7f24940acc00 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2498ad3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24940720c0 con 0x7f2494152340 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2498ad3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2494073060 con 0x7f249414f840 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f249259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24940acc00 0x7f2494147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f249259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24940acc00 0x7f2494147310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58250/0 (socket says 172.21.15.71:58250) 2024-04-16T15:41:22.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f249259c640 1 -- 172.21.15.71:0/4205018721 learned_addr learned my addr 172.21.15.71:0/4205018721 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2492d9d640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2494152340 0x7f24941eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.647+0000 7f2491d9b640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249414f840 0x7f24940c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f249259c640 1 -- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249414f840 msgr2=0x7f24940c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f249259c640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249414f840 0x7f24940c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f249259c640 1 -- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2494152340 msgr2=0x7f24941eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f249259c640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2494152340 0x7f24941eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f249259c640 1 -- 172.21.15.71:0/4205018721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249414c380 con 0x7f24940acc00 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f2491d9b640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249414f840 0x7f24940c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f249259c640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24940acc00 0x7f2494147310 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f248400ba50 tx=0x7f248400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f2482ffd640 1 -- 172.21.15.71:0/4205018721 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484019070 con 0x7f24940acc00 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f2482ffd640 1 -- 172.21.15.71:0/4205018721 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248400cc20 con 0x7f24940acc00 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24941e7610 con 0x7f24940acc00 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f2482ffd640 1 -- 172.21.15.71:0/4205018721 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484014680 con 0x7f24940acc00 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.648+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941e7bd0 con 0x7f24940acc00 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.649+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2494068be0 con 0x7f24940acc00 2024-04-16T15:41:22.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.649+0000 7f2482ffd640 1 -- 172.21.15.71:0/4205018721 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f248401c070 con 0x7f24940acc00 2024-04-16T15:41:22.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.650+0000 7f2482ffd640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f247406c920 0x7f247406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:22.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.650+0000 7f2482ffd640 1 -- 172.21.15.71:0/4205018721 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7f248408b8d0 con 0x7f24940acc00 2024-04-16T15:41:22.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.650+0000 7f2491d9b640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f247406c920 0x7f247406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:22.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.650+0000 7f2491d9b640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f247406c920 0x7f247406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f24940acb20 tx=0x7f2488008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:22.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.654+0000 7f2482ffd640 1 -- 172.21.15.71:0/4205018721 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2484059590 con 0x7f24940acc00 2024-04-16T15:41:22.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:22.866+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f24940c3230 con 0x7f24940acc00 2024-04-16T15:41:23.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.504+0000 7f2482ffd640 1 -- 172.21.15.71:0/4205018721 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f248405d890 con 0x7f24940acc00 2024-04-16T15:41:23.503 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 5 and data pool 6 2024-04-16T15:41:23.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.514+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f247406c920 msgr2=0x7f247406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.514+0000 7f2498ad3640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f247406c920 0x7f247406ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f24940acb20 tx=0x7f2488008040 comp rx=0 tx=0).stop 2024-04-16T15:41:23.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.514+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24940acc00 msgr2=0x7f2494147310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.514+0000 7f2498ad3640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24940acc00 0x7f2494147310 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f248400ba50 tx=0x7f248400bf10 comp rx=0 tx=0).stop 2024-04-16T15:41:23.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.516+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 shutdown_connections 2024-04-16T15:41:23.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.516+0000 7f2498ad3640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f247406c920 0x7f247406ede0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.516+0000 7f2498ad3640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2494152340 0x7f24941eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.516+0000 7f2498ad3640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f249414f840 0x7f24940c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.516+0000 7f2498ad3640 1 --2- 172.21.15.71:0/4205018721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24940acc00 0x7f2494147310 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f248400ba50 tx=0x7f248400bf10 comp rx=0 tx=0).stop 2024-04-16T15:41:23.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.516+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 >> 172.21.15.71:0/4205018721 conn(0x7f24940c7940 msgr2=0x7f249414d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:23.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.519+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 shutdown_connections 2024-04-16T15:41:23.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.519+0000 7f2498ad3640 1 -- 172.21.15.71:0/4205018721 wait complete. 2024-04-16T15:41:23.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:23.592 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:41:23.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:41:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.649+0000 7f7ee6eb3640 1 Processor -- start 2024-04-16T15:41:23.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.649+0000 7f7ee6eb3640 1 -- start start 2024-04-16T15:41:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7ee6eb3640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7ee0153210 0x7f7ee00aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7ee6eb3640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f7ee0156620 0x7f7ee006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7ee6eb3640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee00ab030 con 0x7f7ee006df90 2024-04-16T15:41:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7ee6eb3640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee01535f0 con 0x7f7ee0156620 2024-04-16T15:41:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7ee6eb3640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee00c1c10 con 0x7f7ee0153210 2024-04-16T15:41:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7edffff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f7ee0156620 0x7f7ee006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7edffff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f7ee0156620 0x7f7ee006da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58252/0 (socket says 172.21.15.71:58252) 2024-04-16T15:41:23.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7edffff640 1 -- 172.21.15.71:0/521151020 learned_addr learned my addr 172.21.15.71:0/521151020 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7ee4c4f640 1 --2- 172.21.15.71:0/521151020 >> v2:172.21.15.122:3300/0 conn(0x7f7ee0153210 0x7f7ee00aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7edffff640 1 -- 172.21.15.71:0/521151020 >> v2:172.21.15.122:3300/0 conn(0x7f7ee0153210 msgr2=0x7f7ee00aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7edffff640 1 --2- 172.21.15.71:0/521151020 >> v2:172.21.15.122:3300/0 conn(0x7f7ee0153210 0x7f7ee00aa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7edffff640 1 -- 172.21.15.71:0/521151020 >> v1:172.21.15.122:6789/0 conn(0x7f7ee006df90 legacy=0x7f7ee006fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7edffff640 1 -- 172.21.15.71:0/521151020 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee00c1ef0 con 0x7f7ee0156620 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.650+0000 7f7ee4c4f640 1 --2- 172.21.15.71:0/521151020 >> v2:172.21.15.122:3300/0 conn(0x7f7ee0153210 0x7f7ee00aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.651+0000 7f7edffff640 1 --2- 172.21.15.71:0/521151020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0156620 0x7f7ee006da50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00c5f30 tx=0x7f7ecc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74838070f8f23ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.651+0000 7f7edf7fe640 1 -- 172.21.15.71:0/521151020 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc035020 con 0x7f7ee0156620 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.651+0000 7f7edf7fe640 1 -- 172.21.15.71:0/521151020 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc031870 con 0x7f7ee0156620 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.651+0000 7f7edf7fe640 1 -- 172.21.15.71:0/521151020 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc031c40 con 0x7f7ee0156620 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.652+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/521151020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0156620 msgr2=0x7f7ee006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.652+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/521151020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0156620 0x7f7ee006da50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00c5f30 tx=0x7f7ecc02b910 comp rx=0 tx=0).stop 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.652+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/521151020 shutdown_connections 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.652+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/521151020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0156620 0x7f7ee006da50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.652+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/521151020 >> v2:172.21.15.122:3300/0 conn(0x7f7ee0153210 0x7f7ee00aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.652+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/521151020 >> 172.21.15.71:0/521151020 conn(0x7f7ee0072aa0 msgr2=0x7f7ee0072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.652+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/521151020 shutdown_connections 2024-04-16T15:41:23.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.652+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/521151020 wait complete. 2024-04-16T15:41:23.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.653+0000 7f7ee6eb3640 1 Processor -- start 2024-04-16T15:41:23.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.653+0000 7f7ee6eb3640 1 -- start start 2024-04-16T15:41:23.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee6eb3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee006df90 0x7f7ee01efc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee6eb3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0153210 0x7f7ee0154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee6eb3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ee0156620 0x7f7ee014f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee6eb3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee00712f0 con 0x7f7ee0153210 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee6eb3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0156ab0 con 0x7f7ee006df90 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee6eb3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee01ef770 con 0x7f7ee0156620 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0153210 0x7f7ee0154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0153210 0x7f7ee0154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58264/0 (socket says 172.21.15.71:58264) 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 -- 172.21.15.71:0/3438859196 learned_addr learned my addr 172.21.15.71:0/3438859196 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee4c4f640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee006df90 0x7f7ee01efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee5450640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ee0156620 0x7f7ee014f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 -- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ee0156620 msgr2=0x7f7ee014f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ee0156620 0x7f7ee014f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 -- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee006df90 msgr2=0x7f7ee01efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee006df90 0x7f7ee01efc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 -- 172.21.15.71:0/3438859196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee0153d00 con 0x7f7ee0153210 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee5450640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ee0156620 0x7f7ee014f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edffff640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0153210 0x7f7ee0154f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc02b950 tx=0x7f7ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edd7fa640 1 -- 172.21.15.71:0/3438859196 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc040070 con 0x7f7ee0153210 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edd7fa640 1 -- 172.21.15.71:0/3438859196 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc030d00 con 0x7f7ee0153210 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee01ef160 con 0x7f7ee0153210 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7edd7fa640 1 -- 172.21.15.71:0/3438859196 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc035040 con 0x7f7ee0153210 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.654+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee01ef400 con 0x7f7ee0153210 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.655+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee0068d20 con 0x7f7ee0153210 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.656+0000 7f7edd7fa640 1 -- 172.21.15.71:0/3438859196 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ecc038020 con 0x7f7ee0153210 2024-04-16T15:41:23.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.656+0000 7f7edd7fa640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ec406c8f0 0x7f7ec406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.657+0000 7f7edd7fa640 1 -- 172.21.15.71:0/3438859196 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f7ecc0ad9a0 con 0x7f7ee0153210 2024-04-16T15:41:23.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.657+0000 7f7ee4c4f640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ec406c8f0 0x7f7ec406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.657+0000 7f7ee4c4f640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ec406c8f0 0x7f7ec406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4006fd0 tx=0x7f7ed4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:23.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.661+0000 7f7edd7fa640 1 -- 172.21.15.71:0/3438859196 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ecc07b680 con 0x7f7ee0153210 2024-04-16T15:41:23.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.840+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ee00c2620 con 0x7f7ee0153210 2024-04-16T15:41:23.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.841+0000 7f7edd7fa640 1 -- 172.21.15.71:0/3438859196 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16943 (secure 0 0 0) 0x7f7ecc07f980 con 0x7f7ee0153210 2024-04-16T15:41:23.841 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:23.841 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":37,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:22.868017+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:41:21.360227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-16T15:41:21.816837+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.843+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ec406c8f0 msgr2=0x7f7ec406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.843+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ec406c8f0 0x7f7ec406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4006fd0 tx=0x7f7ed4008040 comp rx=0 tx=0).stop 2024-04-16T15:41:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0153210 msgr2=0x7f7ee0154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0153210 0x7f7ee0154f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc02b950 tx=0x7f7ecc0046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 shutdown_connections 2024-04-16T15:41:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7ec406c8f0 0x7f7ec406edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ee0156620 0x7f7ee014f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ee0153210 0x7f7ee0154f70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 --2- 172.21.15.71:0/3438859196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee006df90 0x7f7ee01efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 >> 172.21.15.71:0/3438859196 conn(0x7f7ee0072aa0 msgr2=0x7f7ee0152380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 shutdown_connections 2024-04-16T15:41:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.844+0000 7f7ee6eb3640 1 -- 172.21.15.71:0/3438859196 wait complete. 2024-04-16T15:41:23.855 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.959+0000 7f5b6e374640 1 Processor -- start 2024-04-16T15:41:23.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.959+0000 7f5b6e374640 1 -- start start 2024-04-16T15:41:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.960+0000 7f5b6e374640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5b680c4350 0x7f5b680c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.960+0000 7f5b6e374640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5b68159740 0x7f5b68159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.960+0000 7f5b6e374640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b680c82d0 con 0x7f5b68153570 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.960+0000 7f5b6e374640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f5b680c41a0 con 0x7f5b680c4350 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.960+0000 7f5b6e374640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f5b68155d70 con 0x7f5b68159740 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.960+0000 7f5b677fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5b680c4350 0x7f5b680c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.960+0000 7f5b677fe640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5b680c4350 0x7f5b680c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58276/0 (socket says 172.21.15.71:58276) 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.960+0000 7f5b677fe640 1 -- 172.21.15.71:0/1421040195 learned_addr learned my addr 172.21.15.71:0/1421040195 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.961+0000 7f5b6c911640 1 --2- 172.21.15.71:0/1421040195 >> v2:172.21.15.122:3300/0 conn(0x7f5b68159740 0x7f5b68159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.961+0000 7f5b677fe640 1 -- 172.21.15.71:0/1421040195 >> v2:172.21.15.122:3300/0 conn(0x7f5b68159740 msgr2=0x7f5b68159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.961+0000 7f5b677fe640 1 --2- 172.21.15.71:0/1421040195 >> v2:172.21.15.122:3300/0 conn(0x7f5b68159740 0x7f5b68159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.961+0000 7f5b677fe640 1 -- 172.21.15.71:0/1421040195 >> v1:172.21.15.122:6789/0 conn(0x7f5b68153570 legacy=0x7f5b680c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:23.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.961+0000 7f5b677fe640 1 -- 172.21.15.71:0/1421040195 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b680c8d50 con 0x7f5b680c4350 2024-04-16T15:41:23.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.961+0000 7f5b677fe640 1 --2- 172.21.15.71:0/1421040195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b680c52b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b68153950 tx=0x7f5b5c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3550185d65cc7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:23.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b66ffd640 1 -- 172.21.15.71:0/1421040195 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c038070 con 0x7f5b680c4350 2024-04-16T15:41:23.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b66ffd640 1 -- 172.21.15.71:0/1421040195 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5c02f050 con 0x7f5b680c4350 2024-04-16T15:41:23.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b66ffd640 1 -- 172.21.15.71:0/1421040195 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c031970 con 0x7f5b680c4350 2024-04-16T15:41:23.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b6e374640 1 -- 172.21.15.71:0/1421040195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 msgr2=0x7f5b680c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b6e374640 1 --2- 172.21.15.71:0/1421040195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b680c52b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b68153950 tx=0x7f5b5c02b7f0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b6e374640 1 -- 172.21.15.71:0/1421040195 shutdown_connections 2024-04-16T15:41:23.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b6e374640 1 --2- 172.21.15.71:0/1421040195 >> v2:172.21.15.122:3300/0 conn(0x7f5b68159740 0x7f5b68159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b6e374640 1 --2- 172.21.15.71:0/1421040195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b680c52b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b6e374640 1 -- 172.21.15.71:0/1421040195 >> 172.21.15.71:0/1421040195 conn(0x7f5b680c96e0 msgr2=0x7f5b680c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:23.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b6e374640 1 -- 172.21.15.71:0/1421040195 shutdown_connections 2024-04-16T15:41:23.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.962+0000 7f5b6e374640 1 -- 172.21.15.71:0/1421040195 wait complete. 2024-04-16T15:41:23.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.963+0000 7f5b6e374640 1 Processor -- start 2024-04-16T15:41:23.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b6e374640 1 -- start start 2024-04-16T15:41:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b6e374640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b68158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b6e374640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b68153570 0x7f5b6814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b6e374640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68159740 0x7f5b680ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b6e374640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6815a110 con 0x7f5b680c4350 2024-04-16T15:41:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b6e374640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b680c8ff0 con 0x7f5b68159740 2024-04-16T15:41:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b6e374640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6815a290 con 0x7f5b68153570 2024-04-16T15:41:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b68158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b68158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58292/0 (socket says 172.21.15.71:58292) 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.964+0000 7f5b67fff640 1 -- 172.21.15.71:0/3949253842 learned_addr learned my addr 172.21.15.71:0/3949253842 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b6c911640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68159740 0x7f5b680ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b677fe640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b68153570 0x7f5b6814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b67fff640 1 -- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b68153570 msgr2=0x7f5b6814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b67fff640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b68153570 0x7f5b6814f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b67fff640 1 -- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68159740 msgr2=0x7f5b680ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b67fff640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68159740 0x7f5b680ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b67fff640 1 -- 172.21.15.71:0/3949253842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b68150c80 con 0x7f5b680c4350 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b6c911640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68159740 0x7f5b680ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:23.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b67fff640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b68158870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800c430 tx=0x7f5b5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b64ff9640 1 -- 172.21.15.71:0/3949253842 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58011790 con 0x7f5b680c4350 2024-04-16T15:41:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b64ff9640 1 -- 172.21.15.71:0/3949253842 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b58017070 con 0x7f5b680c4350 2024-04-16T15:41:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b680a9db0 con 0x7f5b680c4350 2024-04-16T15:41:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b64ff9640 1 -- 172.21.15.71:0/3949253842 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58011bb0 con 0x7f5b680c4350 2024-04-16T15:41:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.965+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b68157120 con 0x7f5b680c4350 2024-04-16T15:41:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.966+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b68068d20 con 0x7f5b680c4350 2024-04-16T15:41:23.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.966+0000 7f5b64ff9640 1 -- 172.21.15.71:0/3949253842 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b5801c460 con 0x7f5b680c4350 2024-04-16T15:41:23.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.967+0000 7f5b64ff9640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b5406c920 0x7f5b5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:23.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.967+0000 7f5b64ff9640 1 -- 172.21.15.71:0/3949253842 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f5b5808bab0 con 0x7f5b680c4350 2024-04-16T15:41:23.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.967+0000 7f5b677fe640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b5406c920 0x7f5b5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:23.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.967+0000 7f5b677fe640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b5406c920 0x7f5b5406ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c035000 tx=0x7f5b5c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:23.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:23.971+0000 7f5b64ff9640 1 -- 172.21.15.71:0/3949253842 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b58059de0 con 0x7f5b680c4350 2024-04-16T15:41:24.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.148+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b680c7c50 con 0x7f5b680c4350 2024-04-16T15:41:24.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.149+0000 7f5b64ff9640 1 -- 172.21.15.71:0/3949253842 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16943 (secure 0 0 0) 0x7f5b580596e0 con 0x7f5b680c4350 2024-04-16T15:41:24.148 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:24.148 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":37,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:22.868017+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:41:21.360227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-16T15:41:21.816837+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:24.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b5406c920 msgr2=0x7f5b5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:24.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b5406c920 0x7f5b5406ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c035000 tx=0x7f5b5c008030 comp rx=0 tx=0).stop 2024-04-16T15:41:24.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 msgr2=0x7f5b68158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:24.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b68158870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800c430 tx=0x7f5b5800c8f0 comp rx=0 tx=0).stop 2024-04-16T15:41:24.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 shutdown_connections 2024-04-16T15:41:24.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b5406c920 0x7f5b5406ede0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:24.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68159740 0x7f5b680ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:24.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b68153570 0x7f5b6814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:24.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 --2- 172.21.15.71:0/3949253842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b680c4350 0x7f5b68158870 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:24.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 >> 172.21.15.71:0/3949253842 conn(0x7f5b680c96e0 msgr2=0x7f5b68155580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:24.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 shutdown_connections 2024-04-16T15:41:24.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.151+0000 7f5b6e374640 1 -- 172.21.15.71:0/3949253842 wait complete. 2024-04-16T15:41:24.161 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-16T15:41:24.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.269+0000 7f110f22c640 1 Processor -- start 2024-04-16T15:41:24.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.270+0000 7f110f22c640 1 -- start start 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.270+0000 7f110f22c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f110814fcd0 0x7f1108071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.270+0000 7f110f22c640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1108072380 con 0x7f11080c3ea0 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.270+0000 7f110f22c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1108155d90 con 0x7f1108156870 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.270+0000 7f110f22c640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f1108155b40 con 0x7f110814fcd0 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.270+0000 7f110cfc8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f110814fcd0 0x7f1108071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.270+0000 7f110d7c9640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f11080c3ea0 0x7f11080c71e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49792/0 (socket says 172.21.15.71:49792) 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.270+0000 7f110d7c9640 1 -- 172.21.15.71:0/2430996169 learned_addr learned my addr 172.21.15.71:0/2430996169 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.271+0000 7f110cfc8640 1 -- 172.21.15.71:0/2430996169 >> v1:172.21.15.122:6789/0 conn(0x7f1108156870 legacy=0x7f11080c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.271+0000 7f110cfc8640 1 -- 172.21.15.71:0/2430996169 >> v1:172.21.15.71:6789/0 conn(0x7f11080c3ea0 legacy=0x7f11080c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:24.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.271+0000 7f11077fe640 1 -- 172.21.15.71:0/2430996169 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156996473 0 0) 0x7f1108072380 con 0x7f11080c3ea0 2024-04-16T15:41:24.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.271+0000 7f110cfc8640 1 -- 172.21.15.71:0/2430996169 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1108156070 con 0x7f110814fcd0 2024-04-16T15:41:24.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.271+0000 7f110cfc8640 1 --2- 172.21.15.71:0/2430996169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f1108071da0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097a0 tx=0x7f10fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2a3c6eed240aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:24.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f11077fe640 1 -- 172.21.15.71:0/2430996169 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0315a0 con 0x7f110814fcd0 2024-04-16T15:41:24.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f11077fe640 1 -- 172.21.15.71:0/2430996169 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10fc037070 con 0x7f110814fcd0 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f11077fe640 1 -- 172.21.15.71:0/2430996169 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc031960 con 0x7f110814fcd0 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f110f22c640 1 -- 172.21.15.71:0/2430996169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 msgr2=0x7f1108071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f110f22c640 1 --2- 172.21.15.71:0/2430996169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f1108071da0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097a0 tx=0x7f10fc02b840 comp rx=0 tx=0).stop 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f110f22c640 1 -- 172.21.15.71:0/2430996169 shutdown_connections 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f110f22c640 1 --2- 172.21.15.71:0/2430996169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f1108071da0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f110f22c640 1 -- 172.21.15.71:0/2430996169 >> 172.21.15.71:0/2430996169 conn(0x7f11080ab270 msgr2=0x7f11080ab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f110f22c640 1 -- 172.21.15.71:0/2430996169 shutdown_connections 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.272+0000 7f110f22c640 1 -- 172.21.15.71:0/2430996169 wait complete. 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.273+0000 7f110f22c640 1 Processor -- start 2024-04-16T15:41:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.273+0000 7f110f22c640 1 -- start start 2024-04-16T15:41:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f110f22c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11080c3ea0 0x7f1108152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f110f22c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f110814ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f110f22c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1108156870 0x7f1108155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f110f22c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11080c2e20 con 0x7f110814fcd0 2024-04-16T15:41:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f110f22c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11080c9000 con 0x7f11080c3ea0 2024-04-16T15:41:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f110f22c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f11080c2fa0 con 0x7f1108156870 2024-04-16T15:41:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f1107fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f110814ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f1107fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f110814ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58320/0 (socket says 172.21.15.71:58320) 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f1107fff640 1 -- 172.21.15.71:0/425323352 learned_addr learned my addr 172.21.15.71:0/425323352 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f110d7c9640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1108156870 0x7f1108155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.274+0000 7f110cfc8640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11080c3ea0 0x7f1108152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f1107fff640 1 -- 172.21.15.71:0/425323352 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1108156870 msgr2=0x7f1108155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f1107fff640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1108156870 0x7f1108155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f1107fff640 1 -- 172.21.15.71:0/425323352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11080c3ea0 msgr2=0x7f1108152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f1107fff640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11080c3ea0 0x7f1108152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f1107fff640 1 -- 172.21.15.71:0/425323352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110814e260 con 0x7f110814fcd0 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f1107fff640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f110814ed50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c480 tx=0x7f10f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f11057fa640 1 -- 172.21.15.71:0/425323352 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40052d0 con 0x7f110814fcd0 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f11057fa640 1 -- 172.21.15.71:0/425323352 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f4005470 con 0x7f110814fcd0 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f11057fa640 1 -- 172.21.15.71:0/425323352 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4012870 con 0x7f110814fcd0 2024-04-16T15:41:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11081572d0 con 0x7f110814fcd0 2024-04-16T15:41:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.275+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081575b0 con 0x7f110814fcd0 2024-04-16T15:41:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.276+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11080c5750 con 0x7f110814fcd0 2024-04-16T15:41:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.276+0000 7f11057fa640 1 -- 172.21.15.71:0/425323352 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10f4012a10 con 0x7f110814fcd0 2024-04-16T15:41:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.277+0000 7f11057fa640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f10ec06c920 0x7f10ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.277+0000 7f11057fa640 1 -- 172.21.15.71:0/425323352 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f10f408b7c0 con 0x7f110814fcd0 2024-04-16T15:41:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.277+0000 7f110cfc8640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f10ec06c920 0x7f10ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.277+0000 7f110cfc8640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f10ec06c920 0x7f10ec06ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10fc036000 tx=0x7f10fc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:24.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.281+0000 7f11057fa640 1 -- 172.21.15.71:0/425323352 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11080c5750 con 0x7f110814fcd0 2024-04-16T15:41:24.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:24.492+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1108068be0 con 0x7f110814fcd0 2024-04-16T15:41:24.506 INFO:tasks.ceph.mds.a.smithi071.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2868-gc3c87ed0/rpm/el9/BUILD/ceph-19.0.0-2868-gc3c87ed0/src/mds/MDLog.cc: In function 'void MDLog::trim()' thread 7f985acdc640 time 2024-04-16T15:41:24.509214+0000 2024-04-16T15:41:24.507 INFO:tasks.ceph.mds.a.smithi071.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2868-gc3c87ed0/rpm/el9/BUILD/ceph-19.0.0-2868-gc3c87ed0/src/mds/MDLog.cc: 688: FAILED ceph_assert(num_total_segments >= num_replayed_segments) 2024-04-16T15:41:24.507 INFO:tasks.ceph.mds.a.smithi071.stderr: ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev) 2024-04-16T15:41:24.507 INFO:tasks.ceph.mds.a.smithi071.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f9865181f4c] 2024-04-16T15:41:24.507 INFO:tasks.ceph.mds.a.smithi071.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18210a) [0x7f986518210a] 2024-04-16T15:41:24.507 INFO:tasks.ceph.mds.a.smithi071.stderr: 3: (MDLog::trim()+0xf2a) [0x556760faa1aa] 2024-04-16T15:41:24.507 INFO:tasks.ceph.mds.a.smithi071.stderr: 4: (MDLog::log_trim_upkeep()+0x339) [0x556760faa559] 2024-04-16T15:41:24.507 INFO:tasks.ceph.mds.a.smithi071.stderr: 5: /lib64/libstdc++.so.6(+0xdb924) [0x7f9864cdb924] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 6: /lib64/libc.so.6(+0x9f802) [0x7f986489f802] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 7: /lib64/libc.so.6(+0x3f450) [0x7f986483f450] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr:2024-04-16T15:41:24.509+0000 7f985acdc640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2868-gc3c87ed0/rpm/el9/BUILD/ceph-19.0.0-2868-gc3c87ed0/src/mds/MDLog.cc: In function 'void MDLog::trim()' thread 7f985acdc640 time 2024-04-16T15:41:24.509214+0000 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2868-gc3c87ed0/rpm/el9/BUILD/ceph-19.0.0-2868-gc3c87ed0/src/mds/MDLog.cc: 688: FAILED ceph_assert(num_total_segments >= num_replayed_segments) 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev) 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f9865181f4c] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18210a) [0x7f986518210a] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 3: (MDLog::trim()+0xf2a) [0x556760faa1aa] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 4: (MDLog::log_trim_upkeep()+0x339) [0x556760faa559] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 5: /lib64/libstdc++.so.6(+0xdb924) [0x7f9864cdb924] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 6: /lib64/libc.so.6(+0x9f802) [0x7f986489f802] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 7: /lib64/libc.so.6(+0x3f450) [0x7f986483f450] 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.508 INFO:tasks.ceph.mds.a.smithi071.stderr:*** Caught signal (Aborted) ** 2024-04-16T15:41:24.509 INFO:tasks.ceph.mds.a.smithi071.stderr: in thread 7f985acdc640 thread_name:ms_dispatch 2024-04-16T15:41:24.509 INFO:tasks.ceph.mds.a.smithi071.stderr: ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev) 2024-04-16T15:41:24.509 INFO:tasks.ceph.mds.a.smithi071.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f9864854db0] 2024-04-16T15:41:24.509 INFO:tasks.ceph.mds.a.smithi071.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f98648a154c] 2024-04-16T15:41:24.509 INFO:tasks.ceph.mds.a.smithi071.stderr: 3: raise() 2024-04-16T15:41:24.509 INFO:tasks.ceph.mds.a.smithi071.stderr: 4: abort() 2024-04-16T15:41:24.509 INFO:tasks.ceph.mds.a.smithi071.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f9865181fa6] 2024-04-16T15:41:24.509 INFO:tasks.ceph.mds.a.smithi071.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18210a) [0x7f986518210a] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 7: (MDLog::trim()+0xf2a) [0x556760faa1aa] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 8: (MDLog::log_trim_upkeep()+0x339) [0x556760faa559] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 9: /lib64/libstdc++.so.6(+0xdb924) [0x7f9864cdb924] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f986489f802] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f986483f450] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr:2024-04-16T15:41:24.509+0000 7f985acdc640 -1 *** Caught signal (Aborted) ** 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: in thread 7f985acdc640 thread_name:ms_dispatch 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev) 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f9864854db0] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f98648a154c] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 3: raise() 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 4: abort() 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f9865181fa6] 2024-04-16T15:41:24.510 INFO:tasks.ceph.mds.a.smithi071.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18210a) [0x7f986518210a] 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: 7: (MDLog::trim()+0xf2a) [0x556760faa1aa] 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: 8: (MDLog::log_trim_upkeep()+0x339) [0x556760faa559] 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: 9: /lib64/libstdc++.so.6(+0xdb924) [0x7f9864cdb924] 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f986489f802] 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f986483f450] 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: -1> 2024-04-16T15:41:24.509+0000 7f985acdc640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2868-gc3c87ed0/rpm/el9/BUILD/ceph-19.0.0-2868-gc3c87ed0/src/mds/MDLog.cc: In function 'void MDLog::trim()' thread 7f985acdc640 time 2024-04-16T15:41:24.509214+0000 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2868-gc3c87ed0/rpm/el9/BUILD/ceph-19.0.0-2868-gc3c87ed0/src/mds/MDLog.cc: 688: FAILED ceph_assert(num_total_segments >= num_replayed_segments) 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.511 INFO:tasks.ceph.mds.a.smithi071.stderr: ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev) 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f9865181f4c] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18210a) [0x7f986518210a] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 3: (MDLog::trim()+0xf2a) [0x556760faa1aa] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 4: (MDLog::log_trim_upkeep()+0x339) [0x556760faa559] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 5: /lib64/libstdc++.so.6(+0xdb924) [0x7f9864cdb924] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 6: /lib64/libc.so.6(+0x9f802) [0x7f986489f802] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 7: /lib64/libc.so.6(+0x3f450) [0x7f986483f450] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 0> 2024-04-16T15:41:24.509+0000 7f985acdc640 -1 *** Caught signal (Aborted) ** 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: in thread 7f985acdc640 thread_name:ms_dispatch 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev) 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f9864854db0] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f98648a154c] 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 3: raise() 2024-04-16T15:41:24.512 INFO:tasks.ceph.mds.a.smithi071.stderr: 4: abort() 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f9865181fa6] 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18210a) [0x7f986518210a] 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 7: (MDLog::trim()+0xf2a) [0x556760faa1aa] 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 8: (MDLog::log_trim_upkeep()+0x339) [0x556760faa559] 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 9: /lib64/libstdc++.so.6(+0xdb924) [0x7f9864cdb924] 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f986489f802] 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f986483f450] 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: -1> 2024-04-16T15:41:24.509+0000 7f985acdc640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2868-gc3c87ed0/rpm/el9/BUILD/ceph-19.0.0-2868-gc3c87ed0/src/mds/MDLog.cc: In function 'void MDLog::trim()' thread 7f985acdc640 time 2024-04-16T15:41:24.509214+0000 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2868-gc3c87ed0/rpm/el9/BUILD/ceph-19.0.0-2868-gc3c87ed0/src/mds/MDLog.cc: 688: FAILED ceph_assert(num_total_segments >= num_replayed_segments) 2024-04-16T15:41:24.513 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev) 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f9865181f4c] 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18210a) [0x7f986518210a] 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 3: (MDLog::trim()+0xf2a) [0x556760faa1aa] 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 4: (MDLog::log_trim_upkeep()+0x339) [0x556760faa559] 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 5: /lib64/libstdc++.so.6(+0xdb924) [0x7f9864cdb924] 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 6: /lib64/libc.so.6(+0x9f802) [0x7f986489f802] 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 7: /lib64/libc.so.6(+0x3f450) [0x7f986483f450] 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 0> 2024-04-16T15:41:24.509+0000 7f985acdc640 -1 *** Caught signal (Aborted) ** 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: in thread 7f985acdc640 thread_name:ms_dispatch 2024-04-16T15:41:24.514 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: ceph version 19.0.0-2868-gc3c87ed0 (c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f) squid (dev) 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f9864854db0] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f98648a154c] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 3: raise() 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 4: abort() 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f9865181fa6] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18210a) [0x7f986518210a] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 7: (MDLog::trim()+0xf2a) [0x556760faa1aa] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 8: (MDLog::log_trim_upkeep()+0x339) [0x556760faa559] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 9: /lib64/libstdc++.so.6(+0xdb924) [0x7f9864cdb924] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f986489f802] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f986483f450] 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-16T15:41:24.515 INFO:tasks.ceph.mds.a.smithi071.stderr: 2024-04-16T15:41:24.812 INFO:tasks.ceph.mds.a.smithi071.stderr:daemon-helper: command crashed with signal 6 2024-04-16T15:41:25.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.266+0000 7f11057fa640 1 -- 172.21.15.71:0/425323352 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f10f40594a0 con 0x7f110814fcd0 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f10ec06c920 msgr2=0x7f10ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f10ec06c920 0x7f10ec06ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10fc036000 tx=0x7f10fc03d040 comp rx=0 tx=0).stop 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 msgr2=0x7f110814ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f110814ed50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c480 tx=0x7f10f400c940 comp rx=0 tx=0).stop 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 shutdown_connections 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f10ec06c920 0x7f10ec06ede0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1108156870 0x7f1108155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f110814fcd0 0x7f110814ed50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 --2- 172.21.15.71:0/425323352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11080c3ea0 0x7f1108152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 >> 172.21.15.71:0/425323352 conn(0x7f11080ab270 msgr2=0x7f110814bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 shutdown_connections 2024-04-16T15:41:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.269+0000 7f110f22c640 1 -- 172.21.15.71:0/425323352 wait complete. 2024-04-16T15:41:25.281 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-16T15:41:25.281 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-16T15:41:25.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.394+0000 7fa113d15640 1 Processor -- start 2024-04-16T15:41:25.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.394+0000 7fa113d15640 1 -- start start 2024-04-16T15:41:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa113d15640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa10c06fac0 0x7fa10c0c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa113d15640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10c0c27c0 con 0x7fa10c0ab290 2024-04-16T15:41:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa113d15640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10c0c2aa0 con 0x7fa10c155dc0 2024-04-16T15:41:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa113d15640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fa10c073f20 con 0x7fa10c06fac0 2024-04-16T15:41:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa1112b0640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa10c06fac0 0x7fa10c0c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa1122b2640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fa10c0ab290 0x7fa10c15d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49804/0 (socket says 172.21.15.71:49804) 2024-04-16T15:41:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa1122b2640 1 -- 172.21.15.71:0/1558923938 learned_addr learned my addr 172.21.15.71:0/1558923938 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:25.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa1112b0640 1 -- 172.21.15.71:0/1558923938 >> v1:172.21.15.122:6789/0 conn(0x7fa10c155dc0 legacy=0x7fa10c1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:25.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa1112b0640 1 -- 172.21.15.71:0/1558923938 >> v1:172.21.15.71:6789/0 conn(0x7fa10c0ab290 legacy=0x7fa10c15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:25.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.395+0000 7fa1112b0640 1 -- 172.21.15.71:0/1558923938 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c0c2d80 con 0x7fa10c06fac0 2024-04-16T15:41:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.396+0000 7fa1112b0640 1 --2- 172.21.15.71:0/1558923938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c06fac0 0x7fa10c0c32a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0097a0 tx=0x7fa0fc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7ced30677db9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.396+0000 7fa110aaf640 1 -- 172.21.15.71:0/1558923938 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc034070 con 0x7fa10c06fac0 2024-04-16T15:41:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.396+0000 7fa110aaf640 1 -- 172.21.15.71:0/1558923938 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0fc030610 con 0x7fa10c06fac0 2024-04-16T15:41:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.396+0000 7fa110aaf640 1 -- 172.21.15.71:0/1558923938 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0309e0 con 0x7fa10c06fac0 2024-04-16T15:41:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.397+0000 7fa113d15640 1 -- 172.21.15.71:0/1558923938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c06fac0 msgr2=0x7fa10c0c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.397+0000 7fa113d15640 1 --2- 172.21.15.71:0/1558923938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c06fac0 0x7fa10c0c32a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0097a0 tx=0x7fa0fc02b960 comp rx=0 tx=0).stop 2024-04-16T15:41:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.397+0000 7fa113d15640 1 -- 172.21.15.71:0/1558923938 shutdown_connections 2024-04-16T15:41:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.397+0000 7fa113d15640 1 --2- 172.21.15.71:0/1558923938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c06fac0 0x7fa10c0c32a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:25.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.397+0000 7fa113d15640 1 -- 172.21.15.71:0/1558923938 >> 172.21.15.71:0/1558923938 conn(0x7fa10c0c3e10 msgr2=0x7fa10c0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:25.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.397+0000 7fa113d15640 1 -- 172.21.15.71:0/1558923938 shutdown_connections 2024-04-16T15:41:25.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.397+0000 7fa113d15640 1 -- 172.21.15.71:0/1558923938 wait complete. 2024-04-16T15:41:25.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.398+0000 7fa113d15640 1 Processor -- start 2024-04-16T15:41:25.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.398+0000 7fa113d15640 1 -- start start 2024-04-16T15:41:25.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa113d15640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c0ab290 0x7fa10c146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:25.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa113d15640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa10c155dc0 0x7fa10c0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:25.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa113d15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c0c0f70 0x7fa10c141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa113d15640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c06bd70 con 0x7fa10c0ab290 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa113d15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c156ab0 con 0x7fa10c0c0f70 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa113d15640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa10c06bef0 con 0x7fa10c155dc0 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c0ab290 0x7fa10c146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c0ab290 0x7fa10c146730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58334/0 (socket says 172.21.15.71:58334) 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 -- 172.21.15.71:0/3542472072 learned_addr learned my addr 172.21.15.71:0/3542472072 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 -- 172.21.15.71:0/3542472072 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa10c155dc0 msgr2=0x7fa10c0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa10c155dc0 0x7fa10c0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 -- 172.21.15.71:0/3542472072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c0c0f70 msgr2=0x7fa10c141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c0c0f70 0x7fa10c141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 -- 172.21.15.71:0/3542472072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c073650 con 0x7fa10c0ab290 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa111ab1640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c0ab290 0x7fa10c146730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa10800c730 tx=0x7fa10800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa1027fc640 1 -- 172.21.15.71:0/3542472072 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108016020 con 0x7fa10c0ab290 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa1027fc640 1 -- 172.21.15.71:0/3542472072 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa108006d50 con 0x7fa10c0ab290 2024-04-16T15:41:25.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c1ec1d0 con 0x7fa10c0ab290 2024-04-16T15:41:25.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa1027fc640 1 -- 172.21.15.71:0/3542472072 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108003bb0 con 0x7fa10c0ab290 2024-04-16T15:41:25.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.399+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c1ec790 con 0x7fa10c0ab290 2024-04-16T15:41:25.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.400+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10c1438e0 con 0x7fa10c0ab290 2024-04-16T15:41:25.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.401+0000 7fa1027fc640 1 -- 172.21.15.71:0/3542472072 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1080050d0 con 0x7fa10c0ab290 2024-04-16T15:41:25.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.401+0000 7fa1027fc640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0f006c920 0x7fa0f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:25.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.401+0000 7fa1027fc640 1 -- 172.21.15.71:0/3542472072 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa10808b850 con 0x7fa10c0ab290 2024-04-16T15:41:25.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.401+0000 7fa1112b0640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0f006c920 0x7fa0f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:25.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.402+0000 7fa1112b0640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0f006c920 0x7fa0f006ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc036000 tx=0x7fa0fc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:25.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.406+0000 7fa1027fc640 1 -- 172.21.15.71:0/3542472072 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa10c1438e0 con 0x7fa10c0ab290 2024-04-16T15:41:25.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:25.619+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa10c068be0 con 0x7fa10c0ab290 2024-04-16T15:41:26.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.537+0000 7fa1027fc640 1 -- 172.21.15.71:0/3542472072 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fa108020020 con 0x7fa10c0ab290 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0f006c920 msgr2=0x7fa0f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0f006c920 0x7fa0f006ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc036000 tx=0x7fa0fc030000 comp rx=0 tx=0).stop 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c0ab290 msgr2=0x7fa10c146730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c0ab290 0x7fa10c146730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa10800c730 tx=0x7fa10800cbf0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 shutdown_connections 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa0f006c920 0x7fa0f006ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c0c0f70 0x7fa10c141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa10c155dc0 0x7fa10c0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 --2- 172.21.15.71:0/3542472072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa10c0ab290 0x7fa10c146730 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 >> 172.21.15.71:0/3542472072 conn(0x7fa10c0c3e10 msgr2=0x7fa10c142ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:26.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 shutdown_connections 2024-04-16T15:41:26.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.540+0000 7fa113d15640 1 -- 172.21.15.71:0/3542472072 wait complete. 2024-04-16T15:41:26.553 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-16T15:41:26.553 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:41:26.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.663+0000 7f1e3c750640 1 Processor -- start 2024-04-16T15:41:26.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.663+0000 7f1e3c750640 1 -- start start 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.664+0000 7f1e3c750640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1e340ca980 0x7f1e34159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.664+0000 7f1e3c750640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e340ae4b0 con 0x7f1e340c78d0 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.664+0000 7f1e3c750640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e340c6940 con 0x7f1e34159520 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.664+0000 7f1e3c750640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f1e340c9540 con 0x7f1e340ca980 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.664+0000 7f1e3aced640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1e340ca980 0x7f1e34159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.664+0000 7f1e3aced640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1e340ca980 0x7f1e34159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58336/0 (socket says 172.21.15.71:58336) 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.664+0000 7f1e3aced640 1 -- 172.21.15.71:0/4289310020 learned_addr learned my addr 172.21.15.71:0/4289310020 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.664+0000 7f1e39ceb640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f1e340c78d0 0x7f1e340ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49810/0 (socket says 172.21.15.71:49810) 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.665+0000 7f1e3aced640 1 -- 172.21.15.71:0/4289310020 >> v1:172.21.15.122:6789/0 conn(0x7f1e34159520 legacy=0x7f1e3406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.665+0000 7f1e3aced640 1 -- 172.21.15.71:0/4289310020 >> v1:172.21.15.71:6789/0 conn(0x7f1e340c78d0 legacy=0x7f1e340ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.665+0000 7f1e394ea640 1 -- 172.21.15.71:0/4289310020 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484591313 0 0) 0x7f1e340ae4b0 con 0x7f1e340c78d0 2024-04-16T15:41:26.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.665+0000 7f1e3aced640 1 -- 172.21.15.71:0/4289310020 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3415a030 con 0x7f1e340ca980 2024-04-16T15:41:26.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.665+0000 7f1e3aced640 1 --2- 172.21.15.71:0/4289310020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340ca980 0x7f1e34159900 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1e340c96c0 tx=0x7f1e2c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4a103887b6f0bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:26.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e394ea640 1 -- 172.21.15.71:0/4289310020 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0317c0 con 0x7f1e340ca980 2024-04-16T15:41:26.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e394ea640 1 -- 172.21.15.71:0/4289310020 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e2c037070 con 0x7f1e340ca980 2024-04-16T15:41:26.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e394ea640 1 -- 172.21.15.71:0/4289310020 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c031b80 con 0x7f1e340ca980 2024-04-16T15:41:26.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e3c750640 1 -- 172.21.15.71:0/4289310020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340ca980 msgr2=0x7f1e34159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:26.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e3c750640 1 --2- 172.21.15.71:0/4289310020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340ca980 0x7f1e34159900 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1e340c96c0 tx=0x7f1e2c02b860 comp rx=0 tx=0).stop 2024-04-16T15:41:26.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e3c750640 1 -- 172.21.15.71:0/4289310020 shutdown_connections 2024-04-16T15:41:26.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e3c750640 1 --2- 172.21.15.71:0/4289310020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340ca980 0x7f1e34159900 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e3c750640 1 -- 172.21.15.71:0/4289310020 >> 172.21.15.71:0/4289310020 conn(0x7f1e340cb820 msgr2=0x7f1e340cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:26.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e3c750640 1 -- 172.21.15.71:0/4289310020 shutdown_connections 2024-04-16T15:41:26.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.666+0000 7f1e3c750640 1 -- 172.21.15.71:0/4289310020 wait complete. 2024-04-16T15:41:26.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.667+0000 7f1e3c750640 1 Processor -- start 2024-04-16T15:41:26.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.667+0000 7f1e3c750640 1 -- start start 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3c750640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340c78d0 0x7f1e34155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3c750640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e340ca980 0x7f1e341eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3c750640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e34159520 0x7f1e34156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3c750640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e340c9d50 con 0x7f1e340c78d0 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3c750640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e340c9a70 con 0x7f1e34159520 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3c750640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1e340b0cf0 con 0x7f1e340ca980 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3a4ec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340c78d0 0x7f1e34155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3a4ec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340c78d0 0x7f1e34155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58340/0 (socket says 172.21.15.71:58340) 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3a4ec640 1 -- 172.21.15.71:0/2212693382 learned_addr learned my addr 172.21.15.71:0/2212693382 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:26.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3aced640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e34159520 0x7f1e34156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3a4ec640 1 -- 172.21.15.71:0/2212693382 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e340ca980 msgr2=0x7f1e341eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3a4ec640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e340ca980 0x7f1e341eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3a4ec640 1 -- 172.21.15.71:0/2212693382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e34159520 msgr2=0x7f1e34156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3a4ec640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e34159520 0x7f1e34156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.668+0000 7f1e3a4ec640 1 -- 172.21.15.71:0/2212693382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e34154e10 con 0x7f1e340c78d0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.669+0000 7f1e3a4ec640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340c78d0 0x7f1e34155a10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000ba50 tx=0x7f1e3000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.669+0000 7f1e2affd640 1 -- 172.21.15.71:0/2212693382 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30019070 con 0x7f1e340c78d0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.669+0000 7f1e2affd640 1 -- 172.21.15.71:0/2212693382 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3000cbc0 con 0x7f1e340c78d0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.669+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e340c4ec0 con 0x7f1e340c78d0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.669+0000 7f1e2affd640 1 -- 172.21.15.71:0/2212693382 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30014440 con 0x7f1e340c78d0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.669+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e341548f0 con 0x7f1e340c78d0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.670+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e34074060 con 0x7f1e340c78d0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.670+0000 7f1e2affd640 1 -- 172.21.15.71:0/2212693382 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e3001c070 con 0x7f1e340c78d0 2024-04-16T15:41:26.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.670+0000 7f1e2affd640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1e1806c8f0 0x7f1e1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:26.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.671+0000 7f1e39ceb640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1e1806c8f0 0x7f1e1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:26.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.671+0000 7f1e2affd640 1 -- 172.21.15.71:0/2212693382 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1e3008b960 con 0x7f1e340c78d0 2024-04-16T15:41:26.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.671+0000 7f1e39ceb640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1e1806c8f0 0x7f1e1806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e24007fd0 tx=0x7f1e24007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:26.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.675+0000 7f1e2affd640 1 -- 172.21.15.71:0/2212693382 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e30059640 con 0x7f1e340c78d0 2024-04-16T15:41:26.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.889+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e341ef820 con 0x7f1e340c78d0 2024-04-16T15:41:26.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.889+0000 7f1e2affd640 1 -- 172.21.15.71:0/2212693382 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f1e3005d940 con 0x7f1e340c78d0 2024-04-16T15:41:26.888 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:26.889 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6836/4129850104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4129850104},{"type":"v1","addr":"172.21.15.71:6836","nonce":4129850104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2014745378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2014745378},{"type":"v1","addr":"172.21.15.122:6834","nonce":2014745378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1432987242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1432987242},{"type":"v1","addr":"172.21.15.122:6835","nonce":1432987242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:41:22.867981+0000","modified":"2024-04-16T15:41:25.620418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6837/180716660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6835","nonce":180716660},{"type":"v1","addr":"172.21.15.71:6837","nonce":180716660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5199,"qdb_cluster":[5199]},"id":2}]} 2024-04-16T15:41:26.889 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 17 2024-04-16T15:41:26.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1e1806c8f0 msgr2=0x7f1e1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:26.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1e1806c8f0 0x7f1e1806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e24007fd0 tx=0x7f1e24007920 comp rx=0 tx=0).stop 2024-04-16T15:41:26.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340c78d0 msgr2=0x7f1e34155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340c78d0 0x7f1e34155a10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000ba50 tx=0x7f1e3000bf10 comp rx=0 tx=0).stop 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 shutdown_connections 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f1e1806c8f0 0x7f1e1806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e34159520 0x7f1e34156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e340ca980 0x7f1e341eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 --2- 172.21.15.71:0/2212693382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e340c78d0 0x7f1e34155a10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.891+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 >> 172.21.15.71:0/2212693382 conn(0x7f1e340cb820 msgr2=0x7f1e340ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.892+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 shutdown_connections 2024-04-16T15:41:26.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:26.892+0000 7f1e3c750640 1 -- 172.21.15.71:0/2212693382 wait complete. 2024-04-16T15:41:26.902 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:27.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.005+0000 7fde83a0c640 1 Processor -- start 2024-04-16T15:41:27.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.006+0000 7fde83a0c640 1 -- start start 2024-04-16T15:41:27.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.006+0000 7fde83a0c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fde7c159520 0x7fde7c159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.006+0000 7fde83a0c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c0ca270 0x7fde7c0ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.006+0000 7fde83a0c640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c0ae4b0 con 0x7fde7c0c78d0 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.006+0000 7fde83a0c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c0c9540 con 0x7fde7c159520 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.006+0000 7fde83a0c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0c6940 con 0x7fde7c0ca270 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde80fa7640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fde7c0c78d0 0x7fde7c06df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49822/0 (socket says 172.21.15.71:49822) 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde80fa7640 1 -- 172.21.15.71:0/198943383 learned_addr learned my addr 172.21.15.71:0/198943383 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009973348 0 0) 0x7fde7c0ae4b0 con 0x7fde7c0c78d0 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde81fa9640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c0ca270 0x7fde7c0ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde68003520 con 0x7fde7c0c78d0 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde817a8640 1 --2- 172.21.15.71:0/198943383 >> v2:172.21.15.122:3300/0 conn(0x7fde7c159520 0x7fde7c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2353684538 0 0) 0x7fde68003520 con 0x7fde7c0c78d0 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c0ca270 msgr2=0x7fde7c0ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c0ca270 0x7fde7c0ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 >> v2:172.21.15.122:3300/0 conn(0x7fde7c159520 msgr2=0x7fde7c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 --2- 172.21.15.71:0/198943383 >> v2:172.21.15.122:3300/0 conn(0x7fde7c159520 0x7fde7c159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c15a030 con 0x7fde7c0c78d0 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7fde6c001670 con 0x7fde7c0c78d0 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 >> v1:172.21.15.71:6789/0 conn(0x7fde7c0c78d0 legacy=0x7fde7c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c0ca270 0x7fde680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde68003f30 0x7fde68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde68004d40 0x7fde680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0c9540 con 0x7fde68004d40 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0c6940 con 0x7fde68003f30 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fde6800baa0 con 0x7fde7c0ca270 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde81fa9640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde68004d40 0x7fde680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde817a8640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c0ca270 0x7fde680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde80fa7640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde68003f30 0x7fde68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde81fa9640 1 -- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c0ca270 msgr2=0x7fde680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde81fa9640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c0ca270 0x7fde680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde81fa9640 1 -- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde68003f30 msgr2=0x7fde68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.007+0000 7fde81fa9640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde68003f30 0x7fde68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.008+0000 7fde81fa9640 1 -- 172.21.15.71:0/198943383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde68003520 con 0x7fde68004d40 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.008+0000 7fde81fa9640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde68004d40 0x7fde680051c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fde78009ac0 tx=0x7fde68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c9ee2bc141d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.008+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78039070 con 0x7fde68004d40 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.008+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde7802c6b0 con 0x7fde68004d40 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.008+0000 7fde73fff640 1 -- 172.21.15.71:0/198943383 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78009690 con 0x7fde68004d40 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.009+0000 7fde83a0c640 1 -- 172.21.15.71:0/198943383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde68004d40 msgr2=0x7fde680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.009+0000 7fde83a0c640 1 --2- 172.21.15.71:0/198943383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde68004d40 0x7fde680051c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fde78009ac0 tx=0x7fde68005830 comp rx=0 tx=0).stop 2024-04-16T15:41:27.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.009+0000 7fde81fa9640 1 -- 172.21.15.71:0/198943383 reap_dead start 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.009+0000 7fde83a0c640 1 -- 172.21.15.71:0/198943383 shutdown_connections 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.009+0000 7fde83a0c640 1 -- 172.21.15.71:0/198943383 >> 172.21.15.71:0/198943383 conn(0x7fde7c0cb820 msgr2=0x7fde7c0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.009+0000 7fde83a0c640 1 -- 172.21.15.71:0/198943383 shutdown_connections 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.009+0000 7fde83a0c640 1 -- 172.21.15.71:0/198943383 wait complete. 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.010+0000 7fde83a0c640 1 Processor -- start 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.010+0000 7fde83a0c640 1 -- start start 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde83a0c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde7c0af140 0x7fde7c0c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde83a0c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c1ef9d0 0x7fde7c1ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde83a0c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7c150f80 0x7fde7c14fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde83a0c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0c9d50 con 0x7fde7c150f80 2024-04-16T15:41:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde83a0c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0c9a70 con 0x7fde7c0af140 2024-04-16T15:41:27.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde83a0c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0b0cf0 con 0x7fde7c1ef9d0 2024-04-16T15:41:27.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7c150f80 0x7fde7c14fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7c150f80 0x7fde7c14fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58346/0 (socket says 172.21.15.71:58346) 2024-04-16T15:41:27.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 -- 172.21.15.71:0/3219237447 learned_addr learned my addr 172.21.15.71:0/3219237447 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:27.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde817a8640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde7c0af140 0x7fde7c0c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde80fa7640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c1ef9d0 0x7fde7c1ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 -- 172.21.15.71:0/3219237447 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c1ef9d0 msgr2=0x7fde7c1ec2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c1ef9d0 0x7fde7c1ec2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 -- 172.21.15.71:0/3219237447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde7c0af140 msgr2=0x7fde7c0c38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde7c0af140 0x7fde7c0c38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 -- 172.21.15.71:0/3219237447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c155f00 con 0x7fde7c150f80 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.011+0000 7fde81fa9640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7c150f80 0x7fde7c14fee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fde7802c320 tx=0x7fde78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.012+0000 7fde71ffb640 1 -- 172.21.15.71:0/3219237447 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78039070 con 0x7fde7c150f80 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.012+0000 7fde71ffb640 1 -- 172.21.15.71:0/3219237447 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde78043070 con 0x7fde7c150f80 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.012+0000 7fde71ffb640 1 -- 172.21.15.71:0/3219237447 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78009690 con 0x7fde7c150f80 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.012+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c0c28d0 con 0x7fde7c150f80 2024-04-16T15:41:27.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.012+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1ebda0 con 0x7fde7c150f80 2024-04-16T15:41:27.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.013+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c074060 con 0x7fde7c150f80 2024-04-16T15:41:27.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.013+0000 7fde71ffb640 1 -- 172.21.15.71:0/3219237447 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde78003850 con 0x7fde7c150f80 2024-04-16T15:41:27.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.013+0000 7fde71ffb640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fde6006c920 0x7fde6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.014+0000 7fde71ffb640 1 -- 172.21.15.71:0/3219237447 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fde780acb30 con 0x7fde7c150f80 2024-04-16T15:41:27.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.014+0000 7fde817a8640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fde6006c920 0x7fde6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.014+0000 7fde817a8640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fde6006c920 0x7fde6006ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0cb610 tx=0x7fde7400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.018+0000 7fde71ffb640 1 -- 172.21.15.71:0/3219237447 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde7804c020 con 0x7fde7c150f80 2024-04-16T15:41:27.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.194+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde7c1ebfa0 con 0x7fde7c150f80 2024-04-16T15:41:27.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.195+0000 7fde71ffb640 1 -- 172.21.15.71:0/3219237447 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16930 (secure 0 0 0) 0x7fde7c1ebfa0 con 0x7fde7c150f80 2024-04-16T15:41:27.194 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:27.194 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:24.513402+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:41:21.360227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-16T15:41:21.816837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:27.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fde6006c920 msgr2=0x7fde6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fde6006c920 0x7fde6006ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0cb610 tx=0x7fde7400b040 comp rx=0 tx=0).stop 2024-04-16T15:41:27.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7c150f80 msgr2=0x7fde7c14fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7c150f80 0x7fde7c14fee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fde7802c320 tx=0x7fde78004730 comp rx=0 tx=0).stop 2024-04-16T15:41:27.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 shutdown_connections 2024-04-16T15:41:27.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fde6006c920 0x7fde6006ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7c150f80 0x7fde7c14fee0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fde7c1ef9d0 0x7fde7c1ec2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 --2- 172.21.15.71:0/3219237447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde7c0af140 0x7fde7c0c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 >> 172.21.15.71:0/3219237447 conn(0x7fde7c0cb820 msgr2=0x7fde7c0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:27.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 shutdown_connections 2024-04-16T15:41:27.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.197+0000 7fde83a0c640 1 -- 172.21.15.71:0/3219237447 wait complete. 2024-04-16T15:41:27.207 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-16T15:41:27.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:41:27.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.311+0000 7f95cb0b1640 1 Processor -- start 2024-04-16T15:41:27.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.311+0000 7f95cb0b1640 1 -- start start 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95cb0b1640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f95c414f780 0x7f95c406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95cb0b1640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f95c40c35f0 0x7f95c4072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95cb0b1640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c4152060 con 0x7f95c4072b90 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95cb0b1640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f95c4071a30 con 0x7f95c414f780 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95cb0b1640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f95c4071d10 con 0x7f95c40c35f0 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95c8e4d640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f95c414f780 0x7f95c406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95c964e640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f95c4072b90 0x7f95c4155990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:49836/0 (socket says 172.21.15.71:49836) 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95c964e640 1 -- 172.21.15.71:0/616007507 learned_addr learned my addr 172.21.15.71:0/616007507 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:27.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95c3fff640 1 --2- 172.21.15.71:0/616007507 >> v2:172.21.15.122:3300/0 conn(0x7f95c40c35f0 0x7f95c4072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95c37fe640 1 -- 172.21.15.71:0/616007507 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268422714 0 0) 0x7f95c4152060 con 0x7f95c4072b90 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.312+0000 7f95c37fe640 1 -- 172.21.15.71:0/616007507 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95b4003520 con 0x7f95c4072b90 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c8e4d640 1 -- 172.21.15.71:0/616007507 >> v2:172.21.15.122:3300/0 conn(0x7f95c40c35f0 msgr2=0x7f95c4072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c8e4d640 1 --2- 172.21.15.71:0/616007507 >> v2:172.21.15.122:3300/0 conn(0x7f95c40c35f0 0x7f95c4072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c8e4d640 1 -- 172.21.15.71:0/616007507 >> v1:172.21.15.71:6789/0 conn(0x7f95c4072b90 legacy=0x7f95c4155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c8e4d640 1 -- 172.21.15.71:0/616007507 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c4071ff0 con 0x7f95c414f780 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c3fff640 1 --2- 172.21.15.71:0/616007507 >> v2:172.21.15.122:3300/0 conn(0x7f95c40c35f0 0x7f95c4072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c8e4d640 1 --2- 172.21.15.71:0/616007507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c406f930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f95c4073d40 tx=0x7f95b802cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca745b0a10686de server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c37fe640 1 -- 172.21.15.71:0/616007507 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b803a070 con 0x7f95c414f780 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c37fe640 1 -- 172.21.15.71:0/616007507 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b802dad0 con 0x7f95c414f780 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.313+0000 7f95c37fe640 1 -- 172.21.15.71:0/616007507 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b800b040 con 0x7f95c414f780 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.314+0000 7f95cb0b1640 1 -- 172.21.15.71:0/616007507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 msgr2=0x7f95c406f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.314+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/616007507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c406f930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f95c4073d40 tx=0x7f95b802cd20 comp rx=0 tx=0).stop 2024-04-16T15:41:27.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.314+0000 7f95cb0b1640 1 -- 172.21.15.71:0/616007507 shutdown_connections 2024-04-16T15:41:27.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.314+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/616007507 >> v2:172.21.15.122:3300/0 conn(0x7f95c40c35f0 0x7f95c4072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.314+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/616007507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c406f930 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.314+0000 7f95cb0b1640 1 -- 172.21.15.71:0/616007507 >> 172.21.15.71:0/616007507 conn(0x7f95c40c3e10 msgr2=0x7f95c40c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:27.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.314+0000 7f95cb0b1640 1 -- 172.21.15.71:0/616007507 shutdown_connections 2024-04-16T15:41:27.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.314+0000 7f95cb0b1640 1 -- 172.21.15.71:0/616007507 wait complete. 2024-04-16T15:41:27.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.315+0000 7f95cb0b1640 1 Processor -- start 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.315+0000 7f95cb0b1640 1 -- start start 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95cb0b1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95c4072b90 0x7f95c40bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95cb0b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95c40c35f0 0x7f95c41dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95cb0b1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c41e9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95cb0b1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95c406fe70 con 0x7f95c414f780 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95cb0b1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95c414fb60 con 0x7f95c40c35f0 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95cb0b1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f95c41ea2b0 con 0x7f95c4072b90 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95c964e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c41e9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95c964e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c41e9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58372/0 (socket says 172.21.15.71:58372) 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95c964e640 1 -- 172.21.15.71:0/2101151689 learned_addr learned my addr 172.21.15.71:0/2101151689 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95c964e640 1 -- 172.21.15.71:0/2101151689 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95c4072b90 msgr2=0x7f95c40bcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95c964e640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95c4072b90 0x7f95c40bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95c964e640 1 -- 172.21.15.71:0/2101151689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95c40c35f0 msgr2=0x7f95c41dc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95c964e640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95c40c35f0 0x7f95c41dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.316+0000 7f95c964e640 1 -- 172.21.15.71:0/2101151689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c41db710 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.317+0000 7f95c964e640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c41e9d70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00cae0 tx=0x7f95bc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.317+0000 7f95c17fa640 1 -- 172.21.15.71:0/2101151689 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc017070 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.317+0000 7f95c17fa640 1 -- 172.21.15.71:0/2101151689 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc00ac40 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.317+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c40bd2e0 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.317+0000 7f95c17fa640 1 -- 172.21.15.71:0/2101151689 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc012600 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.317+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c40bd5c0 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.318+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c4068d20 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.318+0000 7f95c17fa640 1 -- 172.21.15.71:0/2101151689 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95bc01a070 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.319+0000 7f95c17fa640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95a806c920 0x7f95a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.319+0000 7f95c17fa640 1 -- 172.21.15.71:0/2101151689 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f95bc08b810 con 0x7f95c414f780 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.319+0000 7f95c8e4d640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95a806c920 0x7f95a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.319+0000 7f95c8e4d640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95a806c920 0x7f95a806ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95b8037000 tx=0x7f95b802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.323+0000 7f95c17fa640 1 -- 172.21.15.71:0/2101151689 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95bc0594f0 con 0x7f95c414f780 2024-04-16T15:41:27.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.493+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95c406be80 con 0x7f95a806c920 2024-04-16T15:41:27.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.499+0000 7f95c17fa640 1 -- 172.21.15.71:0/2101151689 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240075 (secure 0 0 0) 0x7f95b807e9e0 con 0x7f95a806c920 2024-04-16T15:41:27.501 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:41:27.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95a806c920 msgr2=0x7f95a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95a806c920 0x7f95a806ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95b8037000 tx=0x7f95b802f000 comp rx=0 tx=0).stop 2024-04-16T15:41:27.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 msgr2=0x7f95c41e9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c41e9d70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00cae0 tx=0x7f95bc00a060 comp rx=0 tx=0).stop 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 shutdown_connections 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f95a806c920 0x7f95a806ede0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414f780 0x7f95c41e9d70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95c40c35f0 0x7f95c41dc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 --2- 172.21.15.71:0/2101151689 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95c4072b90 0x7f95c40bcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 >> 172.21.15.71:0/2101151689 conn(0x7f95c40c3e10 msgr2=0x7f95c414eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 shutdown_connections 2024-04-16T15:41:27.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.502+0000 7f95cb0b1640 1 -- 172.21.15.71:0/2101151689 wait complete. 2024-04-16T15:41:27.512 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-16T15:41:25.707611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220448,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754272,"statfs":{"total":773094113280,"available":772868374528,"internally_reserved":0,"allocated":4300800,"data_stored":2488440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.208664"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503598+0000","last_change":"2024-04-16T15:41:21.503598+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503598+0000","last_undegraded":"2024-04-16T15:41:21.503598+0000","last_fullsized":"2024-04-16T15:41:21.503598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503474+0000","last_change":"2024-04-16T15:41:21.503474+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503474+0000","last_undegraded":"2024-04-16T15:41:21.503474+0000","last_fullsized":"2024-04-16T15:41:21.503474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503968+0000","last_change":"2024-04-16T15:41:21.503968+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503968+0000","last_undegraded":"2024-04-16T15:41:21.503968+0000","last_fullsized":"2024-04-16T15:41:21.503968+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503396+0000","last_change":"2024-04-16T15:41:21.503396+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503396+0000","last_undegraded":"2024-04-16T15:41:21.503396+0000","last_fullsized":"2024-04-16T15:41:21.503396+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.501210+0000","last_change":"2024-04-16T15:41:21.501210+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.501210+0000","last_undegraded":"2024-04-16T15:41:21.501210+0000","last_fullsized":"2024-04-16T15:41:21.501210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.500860+0000","last_change":"2024-04-16T15:41:21.500860+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.500860+0000","last_undegraded":"2024-04-16T15:41:21.500860+0000","last_fullsized":"2024-04-16T15:41:21.500860+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503523+0000","last_change":"2024-04-16T15:41:21.503523+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503523+0000","last_undegraded":"2024-04-16T15:41:21.503523+0000","last_fullsized":"2024-04-16T15:41:21.503523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503831+0000","last_change":"2024-04-16T15:41:21.503831+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503831+0000","last_undegraded":"2024-04-16T15:41:21.503831+0000","last_fullsized":"2024-04-16T15:41:21.503831+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.501365+0000","last_change":"2024-04-16T15:41:21.501365+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.501365+0000","last_undegraded":"2024-04-16T15:41:21.501365+0000","last_fullsized":"2024-04-16T15:41:21.501365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.500900+0000","last_change":"2024-04-16T15:41:21.500900+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.500900+0000","last_undegraded":"2024-04-16T15:41:21.500900+0000","last_fullsized":"2024-04-16T15:41:21.500900+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.500828+0000","last_change":"2024-04-16T15:41:21.500828+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.500828+0000","last_undegraded":"2024-04-16T15:41:21.500828+0000","last_fullsized":"2024-04-16T15:41:21.500828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.501364+0000","last_change":"2024-04-16T15:41:21.501364+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.501364+0000","last_undegraded":"2024-04-16T15:41:21.501364+0000","last_fullsized":"2024-04-16T15:41:21.501364+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.501423+0000","last_change":"2024-04-16T15:41:21.501423+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.501423+0000","last_undegraded":"2024-04-16T15:41:21.501423+0000","last_fullsized":"2024-04-16T15:41:21.501423+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503677+0000","last_change":"2024-04-16T15:41:21.503677+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503677+0000","last_undegraded":"2024-04-16T15:41:21.503677+0000","last_fullsized":"2024-04-16T15:41:21.503677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503551+0000","last_change":"2024-04-16T15:41:21.503551+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503551+0000","last_undegraded":"2024-04-16T15:41:21.503551+0000","last_fullsized":"2024-04-16T15:41:21.503551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.501130+0000","last_change":"2024-04-16T15:41:21.501130+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.501130+0000","last_undegraded":"2024-04-16T15:41:21.501130+0000","last_fullsized":"2024-04-16T15:41:21.501130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.502046+0000","last_change":"2024-04-16T15:41:21.502046+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.502046+0000","last_undegraded":"2024-04-16T15:41:21.502046+0000","last_fullsized":"2024-04-16T15:41:21.502046+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503494+0000","last_change":"2024-04-16T15:41:21.503494+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503494+0000","last_undegraded":"2024-04-16T15:41:21.503494+0000","last_fullsized":"2024-04-16T15:41:21.503494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.505284+0000","last_change":"2024-04-16T15:41:21.505284+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.505284+0000","last_undegraded":"2024-04-16T15:41:21.505284+0000","last_fullsized":"2024-04-16T15:41:21.505284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503690+0000","last_change":"2024-04-16T15:41:21.503690+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503690+0000","last_undegraded":"2024-04-16T15:41:21.503690+0000","last_fullsized":"2024-04-16T15:41:21.503690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.504850+0000","last_change":"2024-04-16T15:41:21.504850+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.504850+0000","last_undegraded":"2024-04-16T15:41:21.504850+0000","last_fullsized":"2024-04-16T15:41:21.504850+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.504756+0000","last_change":"2024-04-16T15:41:21.504756+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.504756+0000","last_undegraded":"2024-04-16T15:41:21.504756+0000","last_fullsized":"2024-04-16T15:41:21.504756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.500916+0000","last_change":"2024-04-16T15:41:21.500916+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.500916+0000","last_undegraded":"2024-04-16T15:41:21.500916+0000","last_fullsized":"2024-04-16T15:41:21.500916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.504875+0000","last_change":"2024-04-16T15:41:21.504875+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.504875+0000","last_undegraded":"2024-04-16T15:41:21.504875+0000","last_fullsized":"2024-04-16T15:41:21.504875+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.501059+0000","last_change":"2024-04-16T15:41:21.501059+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.501059+0000","last_undegraded":"2024-04-16T15:41:21.501059+0000","last_fullsized":"2024-04-16T15:41:21.501059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.504810+0000","last_change":"2024-04-16T15:41:21.504810+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.504810+0000","last_undegraded":"2024-04-16T15:41:21.504810+0000","last_fullsized":"2024-04-16T15:41:21.504810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503482+0000","last_change":"2024-04-16T15:41:21.503482+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503482+0000","last_undegraded":"2024-04-16T15:41:21.503482+0000","last_fullsized":"2024-04-16T15:41:21.503482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503244+0000","last_change":"2024-04-16T15:41:21.503244+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503244+0000","last_undegraded":"2024-04-16T15:41:21.503244+0000","last_fullsized":"2024-04-16T15:41:21.503244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.503958+0000","last_change":"2024-04-16T15:41:21.503958+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.503958+0000","last_undegraded":"2024-04-16T15:41:21.503958+0000","last_fullsized":"2024-04-16T15:41:21.503958+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.504480+0000","last_change":"2024-04-16T15:41:21.504480+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.504480+0000","last_undegraded":"2024-04-16T15:41:21.504480+0000","last_fullsized":"2024-04-16T15:41:21.504480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.501437+0000","last_change":"2024-04-16T15:41:21.501437+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.501437+0000","last_undegraded":"2024-04-16T15:41:21.501437+0000","last_fullsized":"2024-04-16T15:41:21.501437+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-16T15:41:22.498925+0000","last_change":"2024-04-16T15:41:22.498925+0000","last_active":"2024-04-16T15:41:22.498925+0000","last_peered":"2024-04-16T15:41:22.498925+0000","last_clean":"2024-04-16T15:41:22.498925+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:22.498925+0000","last_undegraded":"2024-04-16T15:41:22.498925+0000","last_fullsized":"2024-04-16T15:41:22.498925+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-16T15:41:21.501192+0000","last_change":"2024-04-16T15:41:21.501192+0000","last_active":"2024-04-16T15:41:21.493453+0000","last_peered":"2024-04-16T15:41:21.493453+0000","last_clean":"2024-04-16T15:41:21.493453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-16T15:41:21.501192+0000","last_undegraded":"2024-04-16T15:41:21.501192+0000","last_fullsized":"2024-04-16T15:41:21.501192+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":196,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-16T15:41:21.501368+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:41:21.501368+0000","last_peered":"2024-04-16T15:41:21.501368+0000","last_clean":"2024-04-16T15:41:21.501368+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:41:21.501368+0000","last_undegraded":"2024-04-16T15:41:21.501368+0000","last_fullsized":"2024-04-16T15:41:21.501368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":753831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":163463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":389120,"data_stored":163463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":163463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":163463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":163463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":163463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":753831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:41:27.517 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:41:27.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.657+0000 7f424130e640 1 Processor -- start 2024-04-16T15:41:27.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.657+0000 7f424130e640 1 -- start start 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f424130e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f423c06fc60 0x7f423c154380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f424130e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f423c071750 con 0x7f423c151060 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f424130e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f423c0719f0 con 0x7f423c0c72e0 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f424130e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f423c154990 con 0x7f423c06fc60 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f423affd640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f423c151060 0x7f423c0c4420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35020/0 (socket says 172.21.15.71:35020) 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f423affd640 1 -- 172.21.15.71:0/3009415366 learned_addr learned my addr 172.21.15.71:0/3009415366 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075757500 0 0) 0x7f423c071750 con 0x7f423c151060 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f423a7fc640 1 --2- 172.21.15.71:0/3009415366 >> v2:172.21.15.122:3300/0 conn(0x7f423c06fc60 0x7f423c154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.658+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4224003520 con 0x7f423c151060 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2587805232 0 0) 0x7f4224003520 con 0x7f423c151060 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 >> v2:172.21.15.122:3300/0 conn(0x7f423c06fc60 msgr2=0x7f423c154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:27.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 --2- 172.21.15.71:0/3009415366 >> v2:172.21.15.122:3300/0 conn(0x7f423c06fc60 0x7f423c154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 >> v1:172.21.15.122:6789/0 conn(0x7f423c0c72e0 legacy=0x7f423c15e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c0c8b70 con 0x7f423c151060 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f422c0019f0 con 0x7f423c151060 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 >> v1:172.21.15.71:6789/0 conn(0x7f423c151060 legacy=0x7f423c0c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224003a00 0x7f4224003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4224004800 0x7f4224004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42240051c0 0x7f422400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f423c154990 con 0x7f4224003a00 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f422400c0f0 con 0x7f42240051c0 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f422400c2f0 con 0x7f4224004800 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423affd640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224003a00 0x7f4224003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423b7fe640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42240051c0 0x7f422400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423a7fc640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4224004800 0x7f4224004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423affd640 1 -- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4224004800 msgr2=0x7f4224004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:27.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423affd640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4224004800 0x7f4224004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423affd640 1 -- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42240051c0 msgr2=0x7f422400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423affd640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42240051c0 0x7f422400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423affd640 1 -- 172.21.15.71:0/3009415366 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4224003520 con 0x7f4224003a00 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423b7fe640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42240051c0 0x7f422400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.659+0000 7f423b7fe640 1 -- 172.21.15.71:0/3009415366 reap_dead start 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.660+0000 7f423affd640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224003a00 0x7f4224003e40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f422c00a770 tx=0x7f4224005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e219e3281abcdb9f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.660+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c00a390 con 0x7f4224003a00 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.660+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f422c034650 con 0x7f4224003a00 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.660+0000 7f4239ffb640 1 -- 172.21.15.71:0/3009415366 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c034a40 con 0x7f4224003a00 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.661+0000 7f424130e640 1 -- 172.21.15.71:0/3009415366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224003a00 msgr2=0x7f4224003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.661+0000 7f424130e640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224003a00 0x7f4224003e40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f422c00a770 tx=0x7f4224005830 comp rx=0 tx=0).stop 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.661+0000 7f424130e640 1 -- 172.21.15.71:0/3009415366 shutdown_connections 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.661+0000 7f424130e640 1 --2- 172.21.15.71:0/3009415366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224003a00 0x7f4224003e40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.661+0000 7f424130e640 1 -- 172.21.15.71:0/3009415366 >> 172.21.15.71:0/3009415366 conn(0x7f423c0c93e0 msgr2=0x7f423c0c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:27.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.661+0000 7f424130e640 1 -- 172.21.15.71:0/3009415366 shutdown_connections 2024-04-16T15:41:27.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.661+0000 7f424130e640 1 -- 172.21.15.71:0/3009415366 wait complete. 2024-04-16T15:41:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.662+0000 7f424130e640 1 Processor -- start 2024-04-16T15:41:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.662+0000 7f424130e640 1 -- start start 2024-04-16T15:41:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f424130e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4224003a00 0x7f423c152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f424130e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c1505d0 0x7f423c142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f424130e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c14ee00 0x7f423c149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f424130e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f423c0c9090 con 0x7f423c1505d0 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f424130e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f423c0c9210 con 0x7f423c14ee00 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f424130e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f423c154380 con 0x7f4224003a00 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c1505d0 0x7f423c142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c1505d0 0x7f423c142660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53624/0 (socket says 172.21.15.71:53624) 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 -- 172.21.15.71:0/1479746421 learned_addr learned my addr 172.21.15.71:0/1479746421 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 -- 172.21.15.71:0/1479746421 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4224003a00 msgr2=0x7f423c152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4224003a00 0x7f423c152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 -- 172.21.15.71:0/1479746421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c14ee00 msgr2=0x7f423c149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423b7fe640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c14ee00 0x7f423c149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c14ee00 0x7f423c149ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 -- 172.21.15.71:0/1479746421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c147ea0 con 0x7f423c1505d0 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f423a7fc640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c1505d0 0x7f423c142660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f42300115f0 tx=0x7f4230011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f42237fe640 1 -- 172.21.15.71:0/1479746421 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230015870 con 0x7f423c1505d0 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f42237fe640 1 -- 172.21.15.71:0/1479746421 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4230015a10 con 0x7f423c1505d0 2024-04-16T15:41:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f42237fe640 1 -- 172.21.15.71:0/1479746421 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230015de0 con 0x7f423c1505d0 2024-04-16T15:41:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c1533f0 con 0x7f423c1505d0 2024-04-16T15:41:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.663+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c148a40 con 0x7f423c1505d0 2024-04-16T15:41:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.664+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f423c1e7f00 con 0x7f423c1505d0 2024-04-16T15:41:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.665+0000 7f42237fe640 1 -- 172.21.15.71:0/1479746421 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f423000f480 con 0x7f423c1505d0 2024-04-16T15:41:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.665+0000 7f42237fe640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f422806c920 0x7f422806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.665+0000 7f423affd640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f422806c920 0x7f422806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.665+0000 7f42237fe640 1 -- 172.21.15.71:0/1479746421 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f423008c540 con 0x7f423c1505d0 2024-04-16T15:41:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.666+0000 7f423affd640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f422806c920 0x7f422806ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f422c034000 tx=0x7f422c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:27.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.670+0000 7f42237fe640 1 -- 172.21.15.71:0/1479746421 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4230019030 con 0x7f423c1505d0 2024-04-16T15:41:27.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.842+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f423c068cf0 con 0x7f422806c920 2024-04-16T15:41:27.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.848+0000 7f42237fe640 1 -- 172.21.15.71:0/1479746421 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f423c068cf0 con 0x7f422806c920 2024-04-16T15:41:27.850 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:27.852 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:41:27.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f422806c920 msgr2=0x7f422806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f422806c920 0x7f422806ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f422c034000 tx=0x7f422c007000 comp rx=0 tx=0).stop 2024-04-16T15:41:27.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c1505d0 msgr2=0x7f423c142660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:27.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c1505d0 0x7f423c142660 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f42300115f0 tx=0x7f4230011ab0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 shutdown_connections 2024-04-16T15:41:27.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f422806c920 0x7f422806ede0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c14ee00 0x7f423c149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c1505d0 0x7f423c142660 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 --2- 172.21.15.71:0/1479746421 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4224003a00 0x7f423c152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:27.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 >> 172.21.15.71:0/1479746421 conn(0x7f423c0c93e0 msgr2=0x7f423c0c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:27.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 shutdown_connections 2024-04-16T15:41:27.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:27.851+0000 7f424130e640 1 -- 172.21.15.71:0/1479746421 wait complete. 2024-04-16T15:41:27.861 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-16T15:41:27.708043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220816,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753904,"statfs":{"total":773094113280,"available":772867997696,"internally_reserved":0,"allocated":4759552,"data_stored":2707140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001104"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530182+0000","last_change":"2024-04-16T15:41:23.523079+0000","last_active":"2024-04-16T15:41:24.530182+0000","last_peered":"2024-04-16T15:41:24.530182+0000","last_clean":"2024-04-16T15:41:24.530182+0000","last_became_active":"2024-04-16T15:41:23.522804+0000","last_became_peered":"2024-04-16T15:41:23.522804+0000","last_unstale":"2024-04-16T15:41:24.530182+0000","last_undegraded":"2024-04-16T15:41:24.530182+0000","last_fullsized":"2024-04-16T15:41:24.530182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530026+0000","last_change":"2024-04-16T15:41:23.518347+0000","last_active":"2024-04-16T15:41:24.530026+0000","last_peered":"2024-04-16T15:41:24.530026+0000","last_clean":"2024-04-16T15:41:24.530026+0000","last_became_active":"2024-04-16T15:41:23.517372+0000","last_became_peered":"2024-04-16T15:41:23.517372+0000","last_unstale":"2024-04-16T15:41:24.530026+0000","last_undegraded":"2024-04-16T15:41:24.530026+0000","last_fullsized":"2024-04-16T15:41:24.530026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.533697+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.533520+0000","last_became_peered":"2024-04-16T15:41:23.533520+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069345+0000","last_change":"2024-04-16T15:41:23.521655+0000","last_active":"2024-04-16T15:41:25.069345+0000","last_peered":"2024-04-16T15:41:25.069345+0000","last_clean":"2024-04-16T15:41:25.069345+0000","last_became_active":"2024-04-16T15:41:23.521419+0000","last_became_peered":"2024-04-16T15:41:23.521419+0000","last_unstale":"2024-04-16T15:41:25.069345+0000","last_undegraded":"2024-04-16T15:41:25.069345+0000","last_fullsized":"2024-04-16T15:41:25.069345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:48.607343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069423+0000","last_change":"2024-04-16T15:41:23.522246+0000","last_active":"2024-04-16T15:41:25.069423+0000","last_peered":"2024-04-16T15:41:25.069423+0000","last_clean":"2024-04-16T15:41:25.069423+0000","last_became_active":"2024-04-16T15:41:23.522009+0000","last_became_peered":"2024-04-16T15:41:23.522009+0000","last_unstale":"2024-04-16T15:41:25.069423+0000","last_undegraded":"2024-04-16T15:41:25.069423+0000","last_fullsized":"2024-04-16T15:41:25.069423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:33.728092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069091+0000","last_change":"2024-04-16T15:41:23.523004+0000","last_active":"2024-04-16T15:41:25.069091+0000","last_peered":"2024-04-16T15:41:25.069091+0000","last_clean":"2024-04-16T15:41:25.069091+0000","last_became_active":"2024-04-16T15:41:23.522742+0000","last_became_peered":"2024-04-16T15:41:23.522742+0000","last_unstale":"2024-04-16T15:41:25.069091+0000","last_undegraded":"2024-04-16T15:41:25.069091+0000","last_fullsized":"2024-04-16T15:41:25.069091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527561+0000","last_change":"2024-04-16T15:41:23.523305+0000","last_active":"2024-04-16T15:41:24.527561+0000","last_peered":"2024-04-16T15:41:24.527561+0000","last_clean":"2024-04-16T15:41:24.527561+0000","last_became_active":"2024-04-16T15:41:23.523022+0000","last_became_peered":"2024-04-16T15:41:23.523022+0000","last_unstale":"2024-04-16T15:41:24.527561+0000","last_undegraded":"2024-04-16T15:41:24.527561+0000","last_fullsized":"2024-04-16T15:41:24.527561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529666+0000","last_change":"2024-04-16T15:41:23.535446+0000","last_active":"2024-04-16T15:41:24.529666+0000","last_peered":"2024-04-16T15:41:24.529666+0000","last_clean":"2024-04-16T15:41:24.529666+0000","last_became_active":"2024-04-16T15:41:23.535156+0000","last_became_peered":"2024-04-16T15:41:23.535156+0000","last_unstale":"2024-04-16T15:41:24.529666+0000","last_undegraded":"2024-04-16T15:41:24.529666+0000","last_fullsized":"2024-04-16T15:41:24.529666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069784+0000","last_change":"2024-04-16T15:41:23.535019+0000","last_active":"2024-04-16T15:41:25.069784+0000","last_peered":"2024-04-16T15:41:25.069784+0000","last_clean":"2024-04-16T15:41:25.069784+0000","last_became_active":"2024-04-16T15:41:23.534726+0000","last_became_peered":"2024-04-16T15:41:23.534726+0000","last_unstale":"2024-04-16T15:41:25.069784+0000","last_undegraded":"2024-04-16T15:41:25.069784+0000","last_fullsized":"2024-04-16T15:41:25.069784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528288+0000","last_change":"2024-04-16T15:41:23.522709+0000","last_active":"2024-04-16T15:41:24.528288+0000","last_peered":"2024-04-16T15:41:24.528288+0000","last_clean":"2024-04-16T15:41:24.528288+0000","last_became_active":"2024-04-16T15:41:23.522437+0000","last_became_peered":"2024-04-16T15:41:23.522437+0000","last_unstale":"2024-04-16T15:41:24.528288+0000","last_undegraded":"2024-04-16T15:41:24.528288+0000","last_fullsized":"2024-04-16T15:41:24.528288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528764+0000","last_change":"2024-04-16T15:41:23.530620+0000","last_active":"2024-04-16T15:41:24.528764+0000","last_peered":"2024-04-16T15:41:24.528764+0000","last_clean":"2024-04-16T15:41:24.528764+0000","last_became_active":"2024-04-16T15:41:23.530320+0000","last_became_peered":"2024-04-16T15:41:23.530320+0000","last_unstale":"2024-04-16T15:41:24.528764+0000","last_undegraded":"2024-04-16T15:41:24.528764+0000","last_fullsized":"2024-04-16T15:41:24.528764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068927+0000","last_change":"2024-04-16T15:41:23.523419+0000","last_active":"2024-04-16T15:41:25.068927+0000","last_peered":"2024-04-16T15:41:25.068927+0000","last_clean":"2024-04-16T15:41:25.068927+0000","last_became_active":"2024-04-16T15:41:23.523081+0000","last_became_peered":"2024-04-16T15:41:23.523081+0000","last_unstale":"2024-04-16T15:41:25.068927+0000","last_undegraded":"2024-04-16T15:41:25.068927+0000","last_fullsized":"2024-04-16T15:41:25.068927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527828+0000","last_change":"2024-04-16T15:41:23.522227+0000","last_active":"2024-04-16T15:41:24.527828+0000","last_peered":"2024-04-16T15:41:24.527828+0000","last_clean":"2024-04-16T15:41:24.527828+0000","last_became_active":"2024-04-16T15:41:23.521531+0000","last_became_peered":"2024-04-16T15:41:23.521531+0000","last_unstale":"2024-04-16T15:41:24.527828+0000","last_undegraded":"2024-04-16T15:41:24.527828+0000","last_fullsized":"2024-04-16T15:41:24.527828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683427+0000","last_change":"2024-04-16T15:41:23.529710+0000","last_active":"2024-04-16T15:41:24.683427+0000","last_peered":"2024-04-16T15:41:24.683427+0000","last_clean":"2024-04-16T15:41:24.683427+0000","last_became_active":"2024-04-16T15:41:23.528557+0000","last_became_peered":"2024-04-16T15:41:23.528557+0000","last_unstale":"2024-04-16T15:41:24.683427+0000","last_undegraded":"2024-04-16T15:41:24.683427+0000","last_fullsized":"2024-04-16T15:41:24.683427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527689+0000","last_change":"2024-04-16T15:41:23.513641+0000","last_active":"2024-04-16T15:41:24.527689+0000","last_peered":"2024-04-16T15:41:24.527689+0000","last_clean":"2024-04-16T15:41:24.527689+0000","last_became_active":"2024-04-16T15:41:23.513340+0000","last_became_peered":"2024-04-16T15:41:23.513340+0000","last_unstale":"2024-04-16T15:41:24.527689+0000","last_undegraded":"2024-04-16T15:41:24.527689+0000","last_fullsized":"2024-04-16T15:41:24.527689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527345+0000","last_change":"2024-04-16T15:41:23.535782+0000","last_active":"2024-04-16T15:41:24.527345+0000","last_peered":"2024-04-16T15:41:24.527345+0000","last_clean":"2024-04-16T15:41:24.527345+0000","last_became_active":"2024-04-16T15:41:23.535549+0000","last_became_peered":"2024-04-16T15:41:23.535549+0000","last_unstale":"2024-04-16T15:41:24.527345+0000","last_undegraded":"2024-04-16T15:41:24.527345+0000","last_fullsized":"2024-04-16T15:41:24.527345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528422+0000","last_change":"2024-04-16T15:41:23.536402+0000","last_active":"2024-04-16T15:41:24.528422+0000","last_peered":"2024-04-16T15:41:24.528422+0000","last_clean":"2024-04-16T15:41:24.528422+0000","last_became_active":"2024-04-16T15:41:23.536009+0000","last_became_peered":"2024-04-16T15:41:23.536009+0000","last_unstale":"2024-04-16T15:41:24.528422+0000","last_undegraded":"2024-04-16T15:41:24.528422+0000","last_fullsized":"2024-04-16T15:41:24.528422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068843+0000","last_change":"2024-04-16T15:41:23.523701+0000","last_active":"2024-04-16T15:41:25.068843+0000","last_peered":"2024-04-16T15:41:25.068843+0000","last_clean":"2024-04-16T15:41:25.068843+0000","last_became_active":"2024-04-16T15:41:23.523442+0000","last_became_peered":"2024-04-16T15:41:23.523442+0000","last_unstale":"2024-04-16T15:41:25.068843+0000","last_undegraded":"2024-04-16T15:41:25.068843+0000","last_fullsized":"2024-04-16T15:41:25.068843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070293+0000","last_change":"2024-04-16T15:41:23.519553+0000","last_active":"2024-04-16T15:41:25.070293+0000","last_peered":"2024-04-16T15:41:25.070293+0000","last_clean":"2024-04-16T15:41:25.070293+0000","last_became_active":"2024-04-16T15:41:23.519258+0000","last_became_peered":"2024-04-16T15:41:23.519258+0000","last_unstale":"2024-04-16T15:41:25.070293+0000","last_undegraded":"2024-04-16T15:41:25.070293+0000","last_fullsized":"2024-04-16T15:41:25.070293+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:51.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068696+0000","last_change":"2024-04-16T15:41:23.522212+0000","last_active":"2024-04-16T15:41:25.068696+0000","last_peered":"2024-04-16T15:41:25.068696+0000","last_clean":"2024-04-16T15:41:25.068696+0000","last_became_active":"2024-04-16T15:41:23.521934+0000","last_became_peered":"2024-04-16T15:41:23.521934+0000","last_unstale":"2024-04-16T15:41:25.068696+0000","last_undegraded":"2024-04-16T15:41:25.068696+0000","last_fullsized":"2024-04-16T15:41:25.068696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:03.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527973+0000","last_change":"2024-04-16T15:41:23.520977+0000","last_active":"2024-04-16T15:41:24.527973+0000","last_peered":"2024-04-16T15:41:24.527973+0000","last_clean":"2024-04-16T15:41:24.527973+0000","last_became_active":"2024-04-16T15:41:23.520677+0000","last_became_peered":"2024-04-16T15:41:23.520677+0000","last_unstale":"2024-04-16T15:41:24.527973+0000","last_undegraded":"2024-04-16T15:41:24.527973+0000","last_fullsized":"2024-04-16T15:41:24.527973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683729+0000","last_change":"2024-04-16T15:41:23.527968+0000","last_active":"2024-04-16T15:41:24.683729+0000","last_peered":"2024-04-16T15:41:24.683729+0000","last_clean":"2024-04-16T15:41:24.683729+0000","last_became_active":"2024-04-16T15:41:23.527690+0000","last_became_peered":"2024-04-16T15:41:23.527690+0000","last_unstale":"2024-04-16T15:41:24.683729+0000","last_undegraded":"2024-04-16T15:41:24.683729+0000","last_fullsized":"2024-04-16T15:41:24.683729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527416+0000","last_change":"2024-04-16T15:41:23.527691+0000","last_active":"2024-04-16T15:41:24.527416+0000","last_peered":"2024-04-16T15:41:24.527416+0000","last_clean":"2024-04-16T15:41:24.527416+0000","last_became_active":"2024-04-16T15:41:23.527372+0000","last_became_peered":"2024-04-16T15:41:23.527372+0000","last_unstale":"2024-04-16T15:41:24.527416+0000","last_undegraded":"2024-04-16T15:41:24.527416+0000","last_fullsized":"2024-04-16T15:41:24.527416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069312+0000","last_change":"2024-04-16T15:41:23.525188+0000","last_active":"2024-04-16T15:41:25.069312+0000","last_peered":"2024-04-16T15:41:25.069312+0000","last_clean":"2024-04-16T15:41:25.069312+0000","last_became_active":"2024-04-16T15:41:23.524939+0000","last_became_peered":"2024-04-16T15:41:23.524939+0000","last_unstale":"2024-04-16T15:41:25.069312+0000","last_undegraded":"2024-04-16T15:41:25.069312+0000","last_fullsized":"2024-04-16T15:41:25.069312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:53.692430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068745+0000","last_change":"2024-04-16T15:41:23.526266+0000","last_active":"2024-04-16T15:41:25.068745+0000","last_peered":"2024-04-16T15:41:25.068745+0000","last_clean":"2024-04-16T15:41:25.068745+0000","last_became_active":"2024-04-16T15:41:23.525999+0000","last_became_peered":"2024-04-16T15:41:23.525999+0000","last_unstale":"2024-04-16T15:41:25.068745+0000","last_undegraded":"2024-04-16T15:41:25.068745+0000","last_fullsized":"2024-04-16T15:41:25.068745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:38.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069155+0000","last_change":"2024-04-16T15:41:23.535525+0000","last_active":"2024-04-16T15:41:25.069155+0000","last_peered":"2024-04-16T15:41:25.069155+0000","last_clean":"2024-04-16T15:41:25.069155+0000","last_became_active":"2024-04-16T15:41:23.535192+0000","last_became_peered":"2024-04-16T15:41:23.535192+0000","last_unstale":"2024-04-16T15:41:25.069155+0000","last_undegraded":"2024-04-16T15:41:25.069155+0000","last_fullsized":"2024-04-16T15:41:25.069155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068809+0000","last_change":"2024-04-16T15:41:23.527164+0000","last_active":"2024-04-16T15:41:25.068809+0000","last_peered":"2024-04-16T15:41:25.068809+0000","last_clean":"2024-04-16T15:41:25.068809+0000","last_became_active":"2024-04-16T15:41:23.526765+0000","last_became_peered":"2024-04-16T15:41:23.526765+0000","last_unstale":"2024-04-16T15:41:25.068809+0000","last_undegraded":"2024-04-16T15:41:25.068809+0000","last_fullsized":"2024-04-16T15:41:25.068809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528932+0000","last_change":"2024-04-16T15:41:23.529704+0000","last_active":"2024-04-16T15:41:24.528932+0000","last_peered":"2024-04-16T15:41:24.528932+0000","last_clean":"2024-04-16T15:41:24.528932+0000","last_became_active":"2024-04-16T15:41:23.525283+0000","last_became_peered":"2024-04-16T15:41:23.525283+0000","last_unstale":"2024-04-16T15:41:24.528932+0000","last_undegraded":"2024-04-16T15:41:24.528932+0000","last_fullsized":"2024-04-16T15:41:24.528932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068718+0000","last_change":"2024-04-16T15:41:23.523057+0000","last_active":"2024-04-16T15:41:25.068718+0000","last_peered":"2024-04-16T15:41:25.068718+0000","last_clean":"2024-04-16T15:41:25.068718+0000","last_became_active":"2024-04-16T15:41:23.522762+0000","last_became_peered":"2024-04-16T15:41:23.522762+0000","last_unstale":"2024-04-16T15:41:25.068718+0000","last_undegraded":"2024-04-16T15:41:25.068718+0000","last_fullsized":"2024-04-16T15:41:25.068718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:57.277773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068960+0000","last_change":"2024-04-16T15:41:23.522445+0000","last_active":"2024-04-16T15:41:25.068960+0000","last_peered":"2024-04-16T15:41:25.068960+0000","last_clean":"2024-04-16T15:41:25.068960+0000","last_became_active":"2024-04-16T15:41:23.522179+0000","last_became_peered":"2024-04-16T15:41:23.522179+0000","last_unstale":"2024-04-16T15:41:25.068960+0000","last_undegraded":"2024-04-16T15:41:25.068960+0000","last_fullsized":"2024-04-16T15:41:25.068960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527557+0000","last_change":"2024-04-16T15:41:23.528582+0000","last_active":"2024-04-16T15:41:24.527557+0000","last_peered":"2024-04-16T15:41:24.527557+0000","last_clean":"2024-04-16T15:41:24.527557+0000","last_became_active":"2024-04-16T15:41:23.528304+0000","last_became_peered":"2024-04-16T15:41:23.528304+0000","last_unstale":"2024-04-16T15:41:24.527557+0000","last_undegraded":"2024-04-16T15:41:24.527557+0000","last_fullsized":"2024-04-16T15:41:24.527557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:58:58.124408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069657+0000","last_change":"2024-04-16T15:41:23.519347+0000","last_active":"2024-04-16T15:41:25.069657+0000","last_peered":"2024-04-16T15:41:25.069657+0000","last_clean":"2024-04-16T15:41:25.069657+0000","last_became_active":"2024-04-16T15:41:23.519092+0000","last_became_peered":"2024-04-16T15:41:23.519092+0000","last_unstale":"2024-04-16T15:41:25.069657+0000","last_undegraded":"2024-04-16T15:41:25.069657+0000","last_fullsized":"2024-04-16T15:41:25.069657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527966+0000","last_change":"2024-04-16T15:41:23.536251+0000","last_active":"2024-04-16T15:41:24.527966+0000","last_peered":"2024-04-16T15:41:24.527966+0000","last_clean":"2024-04-16T15:41:24.527966+0000","last_became_active":"2024-04-16T15:41:23.535963+0000","last_became_peered":"2024-04-16T15:41:23.535963+0000","last_unstale":"2024-04-16T15:41:24.527966+0000","last_undegraded":"2024-04-16T15:41:24.527966+0000","last_fullsized":"2024-04-16T15:41:24.527966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068769+0000","last_change":"2024-04-16T15:41:22.522796+0000","last_active":"2024-04-16T15:41:25.068769+0000","last_peered":"2024-04-16T15:41:25.068769+0000","last_clean":"2024-04-16T15:41:25.068769+0000","last_became_active":"2024-04-16T15:41:22.522542+0000","last_became_peered":"2024-04-16T15:41:22.522542+0000","last_unstale":"2024-04-16T15:41:25.068769+0000","last_undegraded":"2024-04-16T15:41:25.068769+0000","last_fullsized":"2024-04-16T15:41:25.068769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069565+0000","last_change":"2024-04-16T15:41:23.536192+0000","last_active":"2024-04-16T15:41:25.069565+0000","last_peered":"2024-04-16T15:41:25.069565+0000","last_clean":"2024-04-16T15:41:25.069565+0000","last_became_active":"2024-04-16T15:41:23.535920+0000","last_became_peered":"2024-04-16T15:41:23.535920+0000","last_unstale":"2024-04-16T15:41:25.069565+0000","last_undegraded":"2024-04-16T15:41:25.069565+0000","last_fullsized":"2024-04-16T15:41:25.069565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069482+0000","last_change":"2024-04-16T15:41:22.523464+0000","last_active":"2024-04-16T15:41:25.069482+0000","last_peered":"2024-04-16T15:41:25.069482+0000","last_clean":"2024-04-16T15:41:25.069482+0000","last_became_active":"2024-04-16T15:41:22.523128+0000","last_became_peered":"2024-04-16T15:41:22.523128+0000","last_unstale":"2024-04-16T15:41:25.069482+0000","last_undegraded":"2024-04-16T15:41:25.069482+0000","last_fullsized":"2024-04-16T15:41:25.069482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683674+0000","last_change":"2024-04-16T15:41:23.530373+0000","last_active":"2024-04-16T15:41:24.683674+0000","last_peered":"2024-04-16T15:41:24.683674+0000","last_clean":"2024-04-16T15:41:24.683674+0000","last_became_active":"2024-04-16T15:41:23.530115+0000","last_became_peered":"2024-04-16T15:41:23.530115+0000","last_unstale":"2024-04-16T15:41:24.683674+0000","last_undegraded":"2024-04-16T15:41:24.683674+0000","last_fullsized":"2024-04-16T15:41:24.683674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069538+0000","last_change":"2024-04-16T15:41:22.521948+0000","last_active":"2024-04-16T15:41:25.069538+0000","last_peered":"2024-04-16T15:41:25.069538+0000","last_clean":"2024-04-16T15:41:25.069538+0000","last_became_active":"2024-04-16T15:41:22.521643+0000","last_became_peered":"2024-04-16T15:41:22.521643+0000","last_unstale":"2024-04-16T15:41:25.069538+0000","last_undegraded":"2024-04-16T15:41:25.069538+0000","last_fullsized":"2024-04-16T15:41:25.069538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530028+0000","last_change":"2024-04-16T15:41:23.528874+0000","last_active":"2024-04-16T15:41:24.530028+0000","last_peered":"2024-04-16T15:41:24.530028+0000","last_clean":"2024-04-16T15:41:24.530028+0000","last_became_active":"2024-04-16T15:41:23.528569+0000","last_became_peered":"2024-04-16T15:41:23.528569+0000","last_unstale":"2024-04-16T15:41:24.530028+0000","last_undegraded":"2024-04-16T15:41:24.530028+0000","last_fullsized":"2024-04-16T15:41:24.530028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683536+0000","last_change":"2024-04-16T15:41:22.525207+0000","last_active":"2024-04-16T15:41:24.683536+0000","last_peered":"2024-04-16T15:41:24.683536+0000","last_clean":"2024-04-16T15:41:24.683536+0000","last_became_active":"2024-04-16T15:41:22.524864+0000","last_became_peered":"2024-04-16T15:41:22.524864+0000","last_unstale":"2024-04-16T15:41:24.683536+0000","last_undegraded":"2024-04-16T15:41:24.683536+0000","last_fullsized":"2024-04-16T15:41:24.683536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068873+0000","last_change":"2024-04-16T15:41:23.523861+0000","last_active":"2024-04-16T15:41:25.068873+0000","last_peered":"2024-04-16T15:41:25.068873+0000","last_clean":"2024-04-16T15:41:25.068873+0000","last_became_active":"2024-04-16T15:41:23.523587+0000","last_became_peered":"2024-04-16T15:41:23.523587+0000","last_unstale":"2024-04-16T15:41:25.068873+0000","last_undegraded":"2024-04-16T15:41:25.068873+0000","last_fullsized":"2024-04-16T15:41:25.068873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:22.526265+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:22.525849+0000","last_became_peered":"2024-04-16T15:41:22.525849+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:02.400517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528806+0000","last_change":"2024-04-16T15:41:23.521523+0000","last_active":"2024-04-16T15:41:24.528806+0000","last_peered":"2024-04-16T15:41:24.528806+0000","last_clean":"2024-04-16T15:41:24.528806+0000","last_became_active":"2024-04-16T15:41:23.520768+0000","last_became_peered":"2024-04-16T15:41:23.520768+0000","last_unstale":"2024-04-16T15:41:24.528806+0000","last_undegraded":"2024-04-16T15:41:24.528806+0000","last_fullsized":"2024-04-16T15:41:24.528806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527590+0000","last_change":"2024-04-16T15:41:22.528402+0000","last_active":"2024-04-16T15:41:24.527590+0000","last_peered":"2024-04-16T15:41:24.527590+0000","last_clean":"2024-04-16T15:41:24.527590+0000","last_became_active":"2024-04-16T15:41:22.527999+0000","last_became_peered":"2024-04-16T15:41:22.527999+0000","last_unstale":"2024-04-16T15:41:24.527590+0000","last_undegraded":"2024-04-16T15:41:24.527590+0000","last_fullsized":"2024-04-16T15:41:24.527590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683644+0000","last_change":"2024-04-16T15:41:23.535673+0000","last_active":"2024-04-16T15:41:24.683644+0000","last_peered":"2024-04-16T15:41:24.683644+0000","last_clean":"2024-04-16T15:41:24.683644+0000","last_became_active":"2024-04-16T15:41:23.535387+0000","last_became_peered":"2024-04-16T15:41:23.535387+0000","last_unstale":"2024-04-16T15:41:24.683644+0000","last_undegraded":"2024-04-16T15:41:24.683644+0000","last_fullsized":"2024-04-16T15:41:24.683644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070111+0000","last_change":"2024-04-16T15:41:22.526184+0000","last_active":"2024-04-16T15:41:25.070111+0000","last_peered":"2024-04-16T15:41:25.070111+0000","last_clean":"2024-04-16T15:41:25.070111+0000","last_became_active":"2024-04-16T15:41:22.525819+0000","last_became_peered":"2024-04-16T15:41:22.525819+0000","last_unstale":"2024-04-16T15:41:25.070111+0000","last_undegraded":"2024-04-16T15:41:25.070111+0000","last_fullsized":"2024-04-16T15:41:25.070111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530001+0000","last_change":"2024-04-16T15:41:23.522193+0000","last_active":"2024-04-16T15:41:24.530001+0000","last_peered":"2024-04-16T15:41:24.530001+0000","last_clean":"2024-04-16T15:41:24.530001+0000","last_became_active":"2024-04-16T15:41:23.521900+0000","last_became_peered":"2024-04-16T15:41:23.521900+0000","last_unstale":"2024-04-16T15:41:24.530001+0000","last_undegraded":"2024-04-16T15:41:24.530001+0000","last_fullsized":"2024-04-16T15:41:24.530001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.521519+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.521234+0000","last_became_peered":"2024-04-16T15:41:22.521234+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.521070+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.520819+0000","last_became_peered":"2024-04-16T15:41:23.520819+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:03:41.341706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530092+0000","last_change":"2024-04-16T15:41:22.519163+0000","last_active":"2024-04-16T15:41:24.530092+0000","last_peered":"2024-04-16T15:41:24.530092+0000","last_clean":"2024-04-16T15:41:24.530092+0000","last_became_active":"2024-04-16T15:41:22.518905+0000","last_became_peered":"2024-04-16T15:41:22.518905+0000","last_unstale":"2024-04-16T15:41:24.530092+0000","last_undegraded":"2024-04-16T15:41:24.530092+0000","last_fullsized":"2024-04-16T15:41:24.530092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:03.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528974+0000","last_change":"2024-04-16T15:41:23.528995+0000","last_active":"2024-04-16T15:41:24.528974+0000","last_peered":"2024-04-16T15:41:24.528974+0000","last_clean":"2024-04-16T15:41:24.528974+0000","last_became_active":"2024-04-16T15:41:23.526288+0000","last_became_peered":"2024-04-16T15:41:23.526288+0000","last_unstale":"2024-04-16T15:41:24.528974+0000","last_undegraded":"2024-04-16T15:41:24.528974+0000","last_fullsized":"2024-04-16T15:41:24.528974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528459+0000","last_change":"2024-04-16T15:41:22.525433+0000","last_active":"2024-04-16T15:41:24.528459+0000","last_peered":"2024-04-16T15:41:24.528459+0000","last_clean":"2024-04-16T15:41:24.528459+0000","last_became_active":"2024-04-16T15:41:22.524567+0000","last_became_peered":"2024-04-16T15:41:22.524567+0000","last_unstale":"2024-04-16T15:41:24.528459+0000","last_undegraded":"2024-04-16T15:41:24.528459+0000","last_fullsized":"2024-04-16T15:41:24.528459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:56.272300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069430+0000","last_change":"2024-04-16T15:41:23.522367+0000","last_active":"2024-04-16T15:41:25.069430+0000","last_peered":"2024-04-16T15:41:25.069430+0000","last_clean":"2024-04-16T15:41:25.069430+0000","last_became_active":"2024-04-16T15:41:23.521984+0000","last_became_peered":"2024-04-16T15:41:23.521984+0000","last_unstale":"2024-04-16T15:41:25.069430+0000","last_undegraded":"2024-04-16T15:41:25.069430+0000","last_fullsized":"2024-04-16T15:41:25.069430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528812+0000","last_change":"2024-04-16T15:41:22.523086+0000","last_active":"2024-04-16T15:41:24.528812+0000","last_peered":"2024-04-16T15:41:24.528812+0000","last_clean":"2024-04-16T15:41:24.528812+0000","last_became_active":"2024-04-16T15:41:22.522759+0000","last_became_peered":"2024-04-16T15:41:22.522759+0000","last_unstale":"2024-04-16T15:41:24.528812+0000","last_undegraded":"2024-04-16T15:41:24.528812+0000","last_fullsized":"2024-04-16T15:41:24.528812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683778+0000","last_change":"2024-04-16T15:41:23.529819+0000","last_active":"2024-04-16T15:41:24.683778+0000","last_peered":"2024-04-16T15:41:24.683778+0000","last_clean":"2024-04-16T15:41:24.683778+0000","last_became_active":"2024-04-16T15:41:23.529543+0000","last_became_peered":"2024-04-16T15:41:23.529543+0000","last_unstale":"2024-04-16T15:41:24.683778+0000","last_undegraded":"2024-04-16T15:41:24.683778+0000","last_fullsized":"2024-04-16T15:41:24.683778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:22.522411+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:22.522130+0000","last_became_peered":"2024-04-16T15:41:22.522130+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:41.323112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069849+0000","last_change":"2024-04-16T15:41:23.535122+0000","last_active":"2024-04-16T15:41:25.069849+0000","last_peered":"2024-04-16T15:41:25.069849+0000","last_clean":"2024-04-16T15:41:25.069849+0000","last_became_active":"2024-04-16T15:41:23.534825+0000","last_became_peered":"2024-04-16T15:41:23.534825+0000","last_unstale":"2024-04-16T15:41:25.069849+0000","last_undegraded":"2024-04-16T15:41:25.069849+0000","last_fullsized":"2024-04-16T15:41:25.069849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530105+0000","last_change":"2024-04-16T15:41:22.520252+0000","last_active":"2024-04-16T15:41:24.530105+0000","last_peered":"2024-04-16T15:41:24.530105+0000","last_clean":"2024-04-16T15:41:24.530105+0000","last_became_active":"2024-04-16T15:41:22.519829+0000","last_became_peered":"2024-04-16T15:41:22.519829+0000","last_unstale":"2024-04-16T15:41:24.530105+0000","last_undegraded":"2024-04-16T15:41:24.530105+0000","last_fullsized":"2024-04-16T15:41:24.530105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529732+0000","last_change":"2024-04-16T15:41:23.522432+0000","last_active":"2024-04-16T15:41:24.529732+0000","last_peered":"2024-04-16T15:41:24.529732+0000","last_clean":"2024-04-16T15:41:24.529732+0000","last_became_active":"2024-04-16T15:41:23.522159+0000","last_became_peered":"2024-04-16T15:41:23.522159+0000","last_unstale":"2024-04-16T15:41:24.529732+0000","last_undegraded":"2024-04-16T15:41:24.529732+0000","last_fullsized":"2024-04-16T15:41:24.529732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069145+0000","last_change":"2024-04-16T15:41:22.527064+0000","last_active":"2024-04-16T15:41:25.069145+0000","last_peered":"2024-04-16T15:41:25.069145+0000","last_clean":"2024-04-16T15:41:25.069145+0000","last_became_active":"2024-04-16T15:41:22.525811+0000","last_became_peered":"2024-04-16T15:41:22.525811+0000","last_unstale":"2024-04-16T15:41:25.069145+0000","last_undegraded":"2024-04-16T15:41:25.069145+0000","last_fullsized":"2024-04-16T15:41:25.069145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527532+0000","last_change":"2024-04-16T15:41:23.527859+0000","last_active":"2024-04-16T15:41:24.527532+0000","last_peered":"2024-04-16T15:41:24.527532+0000","last_clean":"2024-04-16T15:41:24.527532+0000","last_became_active":"2024-04-16T15:41:23.527295+0000","last_became_peered":"2024-04-16T15:41:23.527295+0000","last_unstale":"2024-04-16T15:41:24.527532+0000","last_undegraded":"2024-04-16T15:41:24.527532+0000","last_fullsized":"2024-04-16T15:41:24.527532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:00:18.934742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068514+0000","last_change":"2024-04-16T15:41:22.524475+0000","last_active":"2024-04-16T15:41:25.068514+0000","last_peered":"2024-04-16T15:41:25.068514+0000","last_clean":"2024-04-16T15:41:25.068514+0000","last_became_active":"2024-04-16T15:41:22.523303+0000","last_became_peered":"2024-04-16T15:41:22.523303+0000","last_unstale":"2024-04-16T15:41:25.068514+0000","last_undegraded":"2024-04-16T15:41:25.068514+0000","last_fullsized":"2024-04-16T15:41:25.068514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:23.523183+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:23.522845+0000","last_became_peered":"2024-04-16T15:41:23.522845+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527394+0000","last_change":"2024-04-16T15:41:22.522941+0000","last_active":"2024-04-16T15:41:24.527394+0000","last_peered":"2024-04-16T15:41:24.527394+0000","last_clean":"2024-04-16T15:41:24.527394+0000","last_became_active":"2024-04-16T15:41:22.522633+0000","last_became_peered":"2024-04-16T15:41:22.522633+0000","last_unstale":"2024-04-16T15:41:24.527394+0000","last_undegraded":"2024-04-16T15:41:24.527394+0000","last_fullsized":"2024-04-16T15:41:24.527394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529991+0000","last_change":"2024-04-16T15:41:23.528140+0000","last_active":"2024-04-16T15:41:24.529991+0000","last_peered":"2024-04-16T15:41:24.529991+0000","last_clean":"2024-04-16T15:41:24.529991+0000","last_became_active":"2024-04-16T15:41:23.527841+0000","last_became_peered":"2024-04-16T15:41:23.527841+0000","last_unstale":"2024-04-16T15:41:24.529991+0000","last_undegraded":"2024-04-16T15:41:24.529991+0000","last_fullsized":"2024-04-16T15:41:24.529991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528233+0000","last_change":"2024-04-16T15:41:22.526961+0000","last_active":"2024-04-16T15:41:24.528233+0000","last_peered":"2024-04-16T15:41:24.528233+0000","last_clean":"2024-04-16T15:41:24.528233+0000","last_became_active":"2024-04-16T15:41:22.526140+0000","last_became_peered":"2024-04-16T15:41:22.526140+0000","last_unstale":"2024-04-16T15:41:24.528233+0000","last_undegraded":"2024-04-16T15:41:24.528233+0000","last_fullsized":"2024-04-16T15:41:24.528233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528839+0000","last_change":"2024-04-16T15:41:23.521298+0000","last_active":"2024-04-16T15:41:24.528839+0000","last_peered":"2024-04-16T15:41:24.528839+0000","last_clean":"2024-04-16T15:41:24.528839+0000","last_became_active":"2024-04-16T15:41:23.521041+0000","last_became_peered":"2024-04-16T15:41:23.521041+0000","last_unstale":"2024-04-16T15:41:24.528839+0000","last_undegraded":"2024-04-16T15:41:24.528839+0000","last_fullsized":"2024-04-16T15:41:24.528839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683715+0000","last_change":"2024-04-16T15:41:22.529046+0000","last_active":"2024-04-16T15:41:24.683715+0000","last_peered":"2024-04-16T15:41:24.683715+0000","last_clean":"2024-04-16T15:41:24.683715+0000","last_became_active":"2024-04-16T15:41:22.528745+0000","last_became_peered":"2024-04-16T15:41:22.528745+0000","last_unstale":"2024-04-16T15:41:24.683715+0000","last_undegraded":"2024-04-16T15:41:24.683715+0000","last_fullsized":"2024-04-16T15:41:24.683715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683671+0000","last_change":"2024-04-16T15:41:23.522758+0000","last_active":"2024-04-16T15:41:24.683671+0000","last_peered":"2024-04-16T15:41:24.683671+0000","last_clean":"2024-04-16T15:41:24.683671+0000","last_became_active":"2024-04-16T15:41:23.521964+0000","last_became_peered":"2024-04-16T15:41:23.521964+0000","last_unstale":"2024-04-16T15:41:24.683671+0000","last_undegraded":"2024-04-16T15:41:24.683671+0000","last_fullsized":"2024-04-16T15:41:24.683671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070256+0000","last_change":"2024-04-16T15:41:22.523228+0000","last_active":"2024-04-16T15:41:25.070256+0000","last_peered":"2024-04-16T15:41:25.070256+0000","last_clean":"2024-04-16T15:41:25.070256+0000","last_became_active":"2024-04-16T15:41:22.522888+0000","last_became_peered":"2024-04-16T15:41:22.522888+0000","last_unstale":"2024-04-16T15:41:25.070256+0000","last_undegraded":"2024-04-16T15:41:25.070256+0000","last_fullsized":"2024-04-16T15:41:25.070256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:59.195492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069536+0000","last_change":"2024-04-16T15:41:23.525797+0000","last_active":"2024-04-16T15:41:25.069536+0000","last_peered":"2024-04-16T15:41:25.069536+0000","last_clean":"2024-04-16T15:41:25.069536+0000","last_became_active":"2024-04-16T15:41:23.525537+0000","last_became_peered":"2024-04-16T15:41:23.525537+0000","last_unstale":"2024-04-16T15:41:25.069536+0000","last_undegraded":"2024-04-16T15:41:25.069536+0000","last_fullsized":"2024-04-16T15:41:25.069536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:06.161198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068884+0000","last_change":"2024-04-16T15:41:22.521885+0000","last_active":"2024-04-16T15:41:25.068884+0000","last_peered":"2024-04-16T15:41:25.068884+0000","last_clean":"2024-04-16T15:41:25.068884+0000","last_became_active":"2024-04-16T15:41:22.521574+0000","last_became_peered":"2024-04-16T15:41:22.521574+0000","last_unstale":"2024-04-16T15:41:25.068884+0000","last_undegraded":"2024-04-16T15:41:25.068884+0000","last_fullsized":"2024-04-16T15:41:25.068884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527576+0000","last_change":"2024-04-16T15:41:23.529208+0000","last_active":"2024-04-16T15:41:24.527576+0000","last_peered":"2024-04-16T15:41:24.527576+0000","last_clean":"2024-04-16T15:41:24.527576+0000","last_became_active":"2024-04-16T15:41:23.528946+0000","last_became_peered":"2024-04-16T15:41:23.528946+0000","last_unstale":"2024-04-16T15:41:24.527576+0000","last_undegraded":"2024-04-16T15:41:24.527576+0000","last_fullsized":"2024-04-16T15:41:24.527576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:59:03.633363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069471+0000","last_change":"2024-04-16T15:41:22.525515+0000","last_active":"2024-04-16T15:41:25.069471+0000","last_peered":"2024-04-16T15:41:25.069471+0000","last_clean":"2024-04-16T15:41:25.069471+0000","last_became_active":"2024-04-16T15:41:22.525228+0000","last_became_peered":"2024-04-16T15:41:22.525228+0000","last_unstale":"2024-04-16T15:41:25.069471+0000","last_undegraded":"2024-04-16T15:41:25.069471+0000","last_fullsized":"2024-04-16T15:41:25.069471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:55:38.421964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068802+0000","last_change":"2024-04-16T15:41:23.522196+0000","last_active":"2024-04-16T15:41:25.068802+0000","last_peered":"2024-04-16T15:41:25.068802+0000","last_clean":"2024-04-16T15:41:25.068802+0000","last_became_active":"2024-04-16T15:41:23.521931+0000","last_became_peered":"2024-04-16T15:41:23.521931+0000","last_unstale":"2024-04-16T15:41:25.068802+0000","last_undegraded":"2024-04-16T15:41:25.068802+0000","last_fullsized":"2024-04-16T15:41:25.068802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069160+0000","last_change":"2024-04-16T15:41:22.523743+0000","last_active":"2024-04-16T15:41:25.069160+0000","last_peered":"2024-04-16T15:41:25.069160+0000","last_clean":"2024-04-16T15:41:25.069160+0000","last_became_active":"2024-04-16T15:41:22.523348+0000","last_became_peered":"2024-04-16T15:41:22.523348+0000","last_unstale":"2024-04-16T15:41:25.069160+0000","last_undegraded":"2024-04-16T15:41:25.069160+0000","last_fullsized":"2024-04-16T15:41:25.069160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:48.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528766+0000","last_change":"2024-04-16T15:41:23.524838+0000","last_active":"2024-04-16T15:41:24.528766+0000","last_peered":"2024-04-16T15:41:24.528766+0000","last_clean":"2024-04-16T15:41:24.528766+0000","last_became_active":"2024-04-16T15:41:23.524540+0000","last_became_peered":"2024-04-16T15:41:23.524540+0000","last_unstale":"2024-04-16T15:41:24.528766+0000","last_undegraded":"2024-04-16T15:41:24.528766+0000","last_fullsized":"2024-04-16T15:41:24.528766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527606+0000","last_change":"2024-04-16T15:41:22.522239+0000","last_active":"2024-04-16T15:41:24.527606+0000","last_peered":"2024-04-16T15:41:24.527606+0000","last_clean":"2024-04-16T15:41:24.527606+0000","last_became_active":"2024-04-16T15:41:22.521860+0000","last_became_peered":"2024-04-16T15:41:22.521860+0000","last_unstale":"2024-04-16T15:41:24.527606+0000","last_undegraded":"2024-04-16T15:41:24.527606+0000","last_fullsized":"2024-04-16T15:41:24.527606+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530187+0000","last_change":"2024-04-16T15:41:23.522134+0000","last_active":"2024-04-16T15:41:24.530187+0000","last_peered":"2024-04-16T15:41:24.530187+0000","last_clean":"2024-04-16T15:41:24.530187+0000","last_became_active":"2024-04-16T15:41:23.521852+0000","last_became_peered":"2024-04-16T15:41:23.521852+0000","last_unstale":"2024-04-16T15:41:24.530187+0000","last_undegraded":"2024-04-16T15:41:24.530187+0000","last_fullsized":"2024-04-16T15:41:24.530187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069255+0000","last_change":"2024-04-16T15:41:22.525713+0000","last_active":"2024-04-16T15:41:25.069255+0000","last_peered":"2024-04-16T15:41:25.069255+0000","last_clean":"2024-04-16T15:41:25.069255+0000","last_became_active":"2024-04-16T15:41:22.525423+0000","last_became_peered":"2024-04-16T15:41:22.525423+0000","last_unstale":"2024-04-16T15:41:25.069255+0000","last_undegraded":"2024-04-16T15:41:25.069255+0000","last_fullsized":"2024-04-16T15:41:25.069255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:22.255097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068828+0000","last_change":"2024-04-16T15:41:23.536118+0000","last_active":"2024-04-16T15:41:25.068828+0000","last_peered":"2024-04-16T15:41:25.068828+0000","last_clean":"2024-04-16T15:41:25.068828+0000","last_became_active":"2024-04-16T15:41:23.535863+0000","last_became_peered":"2024-04-16T15:41:23.535863+0000","last_unstale":"2024-04-16T15:41:25.068828+0000","last_undegraded":"2024-04-16T15:41:25.068828+0000","last_fullsized":"2024-04-16T15:41:25.068828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528405+0000","last_change":"2024-04-16T15:41:22.525073+0000","last_active":"2024-04-16T15:41:24.528405+0000","last_peered":"2024-04-16T15:41:24.528405+0000","last_clean":"2024-04-16T15:41:24.528405+0000","last_became_active":"2024-04-16T15:41:22.524464+0000","last_became_peered":"2024-04-16T15:41:22.524464+0000","last_unstale":"2024-04-16T15:41:24.528405+0000","last_undegraded":"2024-04-16T15:41:24.528405+0000","last_fullsized":"2024-04-16T15:41:24.528405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529022+0000","last_change":"2024-04-16T15:41:23.526852+0000","last_active":"2024-04-16T15:41:24.529022+0000","last_peered":"2024-04-16T15:41:24.529022+0000","last_clean":"2024-04-16T15:41:24.529022+0000","last_became_active":"2024-04-16T15:41:23.526580+0000","last_became_peered":"2024-04-16T15:41:23.526580+0000","last_unstale":"2024-04-16T15:41:24.529022+0000","last_undegraded":"2024-04-16T15:41:24.529022+0000","last_fullsized":"2024-04-16T15:41:24.529022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070264+0000","last_change":"2024-04-16T15:41:22.523803+0000","last_active":"2024-04-16T15:41:25.070264+0000","last_peered":"2024-04-16T15:41:25.070264+0000","last_clean":"2024-04-16T15:41:25.070264+0000","last_became_active":"2024-04-16T15:41:22.523524+0000","last_became_peered":"2024-04-16T15:41:22.523524+0000","last_unstale":"2024-04-16T15:41:25.070264+0000","last_undegraded":"2024-04-16T15:41:25.070264+0000","last_fullsized":"2024-04-16T15:41:25.070264+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:46.457169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530098+0000","last_change":"2024-04-16T15:41:23.521456+0000","last_active":"2024-04-16T15:41:24.530098+0000","last_peered":"2024-04-16T15:41:24.530098+0000","last_clean":"2024-04-16T15:41:24.530098+0000","last_became_active":"2024-04-16T15:41:23.521114+0000","last_became_peered":"2024-04-16T15:41:23.521114+0000","last_unstale":"2024-04-16T15:41:24.530098+0000","last_undegraded":"2024-04-16T15:41:24.530098+0000","last_fullsized":"2024-04-16T15:41:24.530098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069158+0000","last_change":"2024-04-16T15:41:22.517077+0000","last_active":"2024-04-16T15:41:25.069158+0000","last_peered":"2024-04-16T15:41:25.069158+0000","last_clean":"2024-04-16T15:41:25.069158+0000","last_became_active":"2024-04-16T15:41:22.516776+0000","last_became_peered":"2024-04-16T15:41:22.516776+0000","last_unstale":"2024-04-16T15:41:25.069158+0000","last_undegraded":"2024-04-16T15:41:25.069158+0000","last_fullsized":"2024-04-16T15:41:25.069158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070000+0000","last_change":"2024-04-16T15:41:23.521606+0000","last_active":"2024-04-16T15:41:25.070000+0000","last_peered":"2024-04-16T15:41:25.070000+0000","last_clean":"2024-04-16T15:41:25.070000+0000","last_became_active":"2024-04-16T15:41:23.521351+0000","last_became_peered":"2024-04-16T15:41:23.521351+0000","last_unstale":"2024-04-16T15:41:25.070000+0000","last_undegraded":"2024-04-16T15:41:25.070000+0000","last_fullsized":"2024-04-16T15:41:25.070000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069018+0000","last_change":"2024-04-16T15:41:22.517838+0000","last_active":"2024-04-16T15:41:25.069018+0000","last_peered":"2024-04-16T15:41:25.069018+0000","last_clean":"2024-04-16T15:41:25.069018+0000","last_became_active":"2024-04-16T15:41:22.517494+0000","last_became_peered":"2024-04-16T15:41:22.517494+0000","last_unstale":"2024-04-16T15:41:25.069018+0000","last_undegraded":"2024-04-16T15:41:25.069018+0000","last_fullsized":"2024-04-16T15:41:25.069018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:50.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070060+0000","last_change":"2024-04-16T15:41:23.524122+0000","last_active":"2024-04-16T15:41:25.070060+0000","last_peered":"2024-04-16T15:41:25.070060+0000","last_clean":"2024-04-16T15:41:25.070060+0000","last_became_active":"2024-04-16T15:41:23.523821+0000","last_became_peered":"2024-04-16T15:41:23.523821+0000","last_unstale":"2024-04-16T15:41:25.070060+0000","last_undegraded":"2024-04-16T15:41:25.070060+0000","last_fullsized":"2024-04-16T15:41:25.070060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068589+0000","last_change":"2024-04-16T15:41:22.519275+0000","last_active":"2024-04-16T15:41:25.068589+0000","last_peered":"2024-04-16T15:41:25.068589+0000","last_clean":"2024-04-16T15:41:25.068589+0000","last_became_active":"2024-04-16T15:41:22.518999+0000","last_became_peered":"2024-04-16T15:41:22.518999+0000","last_unstale":"2024-04-16T15:41:25.068589+0000","last_undegraded":"2024-04-16T15:41:25.068589+0000","last_fullsized":"2024-04-16T15:41:25.068589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683431+0000","last_change":"2024-04-16T15:41:23.522582+0000","last_active":"2024-04-16T15:41:24.683431+0000","last_peered":"2024-04-16T15:41:24.683431+0000","last_clean":"2024-04-16T15:41:24.683431+0000","last_became_active":"2024-04-16T15:41:23.522254+0000","last_became_peered":"2024-04-16T15:41:23.522254+0000","last_unstale":"2024-04-16T15:41:24.683431+0000","last_undegraded":"2024-04-16T15:41:24.683431+0000","last_fullsized":"2024-04-16T15:41:24.683431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.525440+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.523261+0000","last_became_peered":"2024-04-16T15:41:22.523261+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:23.526801+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:23.526499+0000","last_became_peered":"2024-04-16T15:41:23.526499+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529584+0000","last_change":"2024-04-16T15:41:22.521239+0000","last_active":"2024-04-16T15:41:24.529584+0000","last_peered":"2024-04-16T15:41:24.529584+0000","last_clean":"2024-04-16T15:41:24.529584+0000","last_became_active":"2024-04-16T15:41:22.520944+0000","last_became_peered":"2024-04-16T15:41:22.520944+0000","last_unstale":"2024-04-16T15:41:24.529584+0000","last_undegraded":"2024-04-16T15:41:24.529584+0000","last_fullsized":"2024-04-16T15:41:24.529584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528112+0000","last_change":"2024-04-16T15:41:23.532440+0000","last_active":"2024-04-16T15:41:24.528112+0000","last_peered":"2024-04-16T15:41:24.528112+0000","last_clean":"2024-04-16T15:41:24.528112+0000","last_became_active":"2024-04-16T15:41:23.530755+0000","last_became_peered":"2024-04-16T15:41:23.530755+0000","last_unstale":"2024-04-16T15:41:24.528112+0000","last_undegraded":"2024-04-16T15:41:24.528112+0000","last_fullsized":"2024-04-16T15:41:24.528112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528231+0000","last_change":"2024-04-16T15:41:22.523523+0000","last_active":"2024-04-16T15:41:24.528231+0000","last_peered":"2024-04-16T15:41:24.528231+0000","last_clean":"2024-04-16T15:41:24.528231+0000","last_became_active":"2024-04-16T15:41:22.522766+0000","last_became_peered":"2024-04-16T15:41:22.522766+0000","last_unstale":"2024-04-16T15:41:24.528231+0000","last_undegraded":"2024-04-16T15:41:24.528231+0000","last_fullsized":"2024-04-16T15:41:24.528231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527623+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:41:24.527623+0000","last_peered":"2024-04-16T15:41:24.527623+0000","last_clean":"2024-04-16T15:41:24.527623+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:41:24.527623+0000","last_undegraded":"2024-04-16T15:41:24.527623+0000","last_fullsized":"2024-04-16T15:41:24.527623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":191844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":780600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:41:27.867 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-16T15:41:27.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-16T15:41:28.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.014+0000 7fa1831ba640 1 Processor -- start 2024-04-16T15:41:28.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.014+0000 7fa1831ba640 1 -- start start 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.014+0000 7fa1831ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c153350 0x7fa17c0ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.014+0000 7fa1831ba640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa17c0cb060 0x7fa17c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.014+0000 7fa1831ba640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa17c0c76a0 con 0x7fa17c159960 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.014+0000 7fa1831ba640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fa17c0bee90 con 0x7fa17c0cb060 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.014+0000 7fa1831ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa17c0b0ae0 con 0x7fa17c153350 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa173fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fa17c0cb060 0x7fa17c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa181757640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fa17c159960 0x7fa17c0ca460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35028/0 (socket says 172.21.15.71:35028) 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa181757640 1 -- 172.21.15.71:0/2115575079 learned_addr learned my addr 172.21.15.71:0/2115575079 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa180f56640 1 --2- 172.21.15.71:0/2115575079 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c153350 0x7fa17c0ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa173fff640 1 -- 172.21.15.71:0/2115575079 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c153350 msgr2=0x7fa17c0ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa173fff640 1 --2- 172.21.15.71:0/2115575079 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c153350 0x7fa17c0ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa1737fe640 1 -- 172.21.15.71:0/2115575079 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3758965399 0 0) 0x7fa17c0c76a0 con 0x7fa17c159960 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa173fff640 1 -- 172.21.15.71:0/2115575079 >> v1:172.21.15.71:6789/0 conn(0x7fa17c159960 legacy=0x7fa17c0ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.015+0000 7fa173fff640 1 -- 172.21.15.71:0/2115575079 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c0c7980 con 0x7fa17c0cb060 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa173fff640 1 --2- 172.21.15.71:0/2115575079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c0cb060 0x7fa17c1566a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0097a0 tx=0x7fa16c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c65ed6f7e2cb88 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1737fe640 1 -- 172.21.15.71:0/2115575079 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0315a0 con 0x7fa17c0cb060 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1737fe640 1 -- 172.21.15.71:0/2115575079 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16c037070 con 0x7fa17c0cb060 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1737fe640 1 -- 172.21.15.71:0/2115575079 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c031960 con 0x7fa17c0cb060 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1831ba640 1 -- 172.21.15.71:0/2115575079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c0cb060 msgr2=0x7fa17c1566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1831ba640 1 --2- 172.21.15.71:0/2115575079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c0cb060 0x7fa17c1566a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0097a0 tx=0x7fa16c02b840 comp rx=0 tx=0).stop 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1831ba640 1 -- 172.21.15.71:0/2115575079 shutdown_connections 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1831ba640 1 --2- 172.21.15.71:0/2115575079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c0cb060 0x7fa17c1566a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1831ba640 1 --2- 172.21.15.71:0/2115575079 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c153350 0x7fa17c0ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1831ba640 1 -- 172.21.15.71:0/2115575079 >> 172.21.15.71:0/2115575079 conn(0x7fa17c0ad220 msgr2=0x7fa17c0ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1831ba640 1 -- 172.21.15.71:0/2115575079 shutdown_connections 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.016+0000 7fa1831ba640 1 -- 172.21.15.71:0/2115575079 wait complete. 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.017+0000 7fa1831ba640 1 Processor -- start 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa1831ba640 1 -- start start 2024-04-16T15:41:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa1831ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c0cb060 0x7fa17c0c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa1831ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c153350 0x7fa17c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa1831ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa17c159960 0x7fa17c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa1831ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c159440 con 0x7fa17c153350 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa1831ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c1591f0 con 0x7fa17c159960 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa1831ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa17c0cab70 con 0x7fa17c0cb060 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa173fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c153350 0x7fa17c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa173fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c153350 0x7fa17c155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53638/0 (socket says 172.21.15.71:53638) 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.018+0000 7fa173fff640 1 -- 172.21.15.71:0/3472703032 learned_addr learned my addr 172.21.15.71:0/3472703032 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa173fff640 1 -- 172.21.15.71:0/3472703032 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c0cb060 msgr2=0x7fa17c0c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa180f56640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c0cb060 0x7fa17c0c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa173fff640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c0cb060 0x7fa17c0c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa173fff640 1 -- 172.21.15.71:0/3472703032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa17c159960 msgr2=0x7fa17c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa173fff640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa17c159960 0x7fa17c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa173fff640 1 -- 172.21.15.71:0/3472703032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c0c2a80 con 0x7fa17c153350 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa173fff640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c153350 0x7fa17c155f80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa16c034040 tx=0x7fa16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa1717fa640 1 -- 172.21.15.71:0/3472703032 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c032840 con 0x7fa17c153350 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa1717fa640 1 -- 172.21.15.71:0/3472703032 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16c032a00 con 0x7fa17c153350 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c0c3240 con 0x7fa17c153350 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa1717fa640 1 -- 172.21.15.71:0/3472703032 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c032df0 con 0x7fa17c153350 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.019+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c155580 con 0x7fa17c153350 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.020+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17c074060 con 0x7fa17c153350 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.021+0000 7fa1717fa640 1 -- 172.21.15.71:0/3472703032 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa16c04e430 con 0x7fa17c153350 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.021+0000 7fa1717fa640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa16806c920 0x7fa16806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.021+0000 7fa180f56640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa16806c920 0x7fa16806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.021+0000 7fa1717fa640 1 -- 172.21.15.71:0/3472703032 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa16c02f030 con 0x7fa17c153350 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.022+0000 7fa180f56640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa16806c920 0x7fa16806ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa178006fd0 tx=0x7fa178008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.026+0000 7fa1717fa640 1 -- 172.21.15.71:0/3472703032 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa16c040070 con 0x7fa17c153350 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.265+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa17c0c95d0 con 0x7fa17c153350 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.268+0000 7fa1717fa640 1 -- 172.21.15.71:0/3472703032 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fa16c07bfb0 con 0x7fa17c153350 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQCumx5me0E3GBAAUrYP3hRI128xd+yf1A5VaA== 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2024-04-16T15:41:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa16806c920 msgr2=0x7fa16806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa16806c920 0x7fa16806ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa178006fd0 tx=0x7fa178008040 comp rx=0 tx=0).stop 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c153350 msgr2=0x7fa17c155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c153350 0x7fa17c155f80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa16c034040 tx=0x7fa16c0046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 shutdown_connections 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fa16806c920 0x7fa16806ede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa17c159960 0x7fa17c1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa17c153350 0x7fa17c155f80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.269+0000 7fa1831ba640 1 --2- 172.21.15.71:0/3472703032 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa17c0cb060 0x7fa17c0c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.270+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 >> 172.21.15.71:0/3472703032 conn(0x7fa17c0ad220 msgr2=0x7fa17c0ae900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.270+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 shutdown_connections 2024-04-16T15:41:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.270+0000 7fa1831ba640 1 -- 172.21.15.71:0/3472703032 wait complete. 2024-04-16T15:41:28.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:41:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.409+0000 7f328e86a640 1 Processor -- start 2024-04-16T15:41:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.409+0000 7f328e86a640 1 -- start start 2024-04-16T15:41:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f328e86a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32880c32a0 0x7f328815e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f328e86a640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32880c27c0 con 0x7f3288155dc0 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f328e86a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32880c2aa0 con 0x7f328806fac0 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f328e86a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3288073f20 con 0x7f32880c32a0 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3287fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f3288155dc0 0x7f32881561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35030/0 (socket says 172.21.15.71:35030) 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3287fff640 1 -- 172.21.15.71:0/3960785702 learned_addr learned my addr 172.21.15.71:0/3960785702 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f328ce07640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32880c32a0 0x7f328815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424713480 0 0) 0x7f32880c27c0 con 0x7f3288155dc0 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3270003520 con 0x7f3288155dc0 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911188549 0 0) 0x7f32880c2aa0 con 0x7f328806fac0 2024-04-16T15:41:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32880c27c0 con 0x7f328806fac0 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3603913638 0 0) 0x7f3270003520 con 0x7f3288155dc0 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32880c32a0 msgr2=0x7f328815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32880c32a0 0x7f328815e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 >> v1:172.21.15.122:6789/0 conn(0x7f328806fac0 legacy=0x7f32880ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32880c2d80 con 0x7f3288155dc0 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f3278001670 con 0x7f3288155dc0 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 >> v1:172.21.15.71:6789/0 conn(0x7f3288155dc0 legacy=0x7f32881561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32880c32a0 0x7f3270003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32700040d0 0x7f3270004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f327000ba70 con 0x7f3270004d40 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f327000bc20 con 0x7f32700040d0 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.410+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f327000be20 con 0x7f32880c32a0 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f328ce07640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f3287fff640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32880c32a0 0x7f3270003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f32877fe640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32700040d0 0x7f3270004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f328ce07640 1 -- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32880c32a0 msgr2=0x7f3270003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f328ce07640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32880c32a0 0x7f3270003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f328ce07640 1 -- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32700040d0 msgr2=0x7f3270004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f328ce07640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32700040d0 0x7f3270004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f328ce07640 1 -- 172.21.15.71:0/3960785702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3270003520 con 0x7f3270004d40 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f32877fe640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32700040d0 0x7f3270004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.411+0000 7f328ce07640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32700051c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3280009fc0 tx=0x7f3270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2587c5145318dd93 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:28.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280039070 con 0x7f3270004d40 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328002cc20 con 0x7f3270004d40 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f3286ffd640 1 -- 172.21.15.71:0/3960785702 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280009c10 con 0x7f3270004d40 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f328e86a640 1 -- 172.21.15.71:0/3960785702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 msgr2=0x7f32700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f328e86a640 1 --2- 172.21.15.71:0/3960785702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32700051c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3280009fc0 tx=0x7f3270005830 comp rx=0 tx=0).stop 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f328ce07640 1 -- 172.21.15.71:0/3960785702 reap_dead start 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f328e86a640 1 -- 172.21.15.71:0/3960785702 shutdown_connections 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f328e86a640 1 -- 172.21.15.71:0/3960785702 >> 172.21.15.71:0/3960785702 conn(0x7f32880c3e10 msgr2=0x7f32880c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f328e86a640 1 -- 172.21.15.71:0/3960785702 shutdown_connections 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.412+0000 7f328e86a640 1 -- 172.21.15.71:0/3960785702 wait complete. 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.413+0000 7f328e86a640 1 Processor -- start 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.413+0000 7f328e86a640 1 -- start start 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f328e86a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32880722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f328e86a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32881de520 0x7f32880c09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f328e86a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32881dd770 0x7f32881ddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f328e86a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f32880c18b0 con 0x7f3270004d40 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f328e86a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3288156ab0 con 0x7f32881de520 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f328e86a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f328806bd70 con 0x7f32881dd770 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f3287fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32880722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f3287fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32880722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53650/0 (socket says 172.21.15.71:53650) 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f3287fff640 1 -- 172.21.15.71:0/3039775372 learned_addr learned my addr 172.21.15.71:0/3039775372 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f3287fff640 1 -- 172.21.15.71:0/3039775372 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32881dd770 msgr2=0x7f32881ddbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f3287fff640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32881dd770 0x7f32881ddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f3287fff640 1 -- 172.21.15.71:0/3039775372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32881de520 msgr2=0x7f32880c09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f3287fff640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32881de520 0x7f32880c09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.414+0000 7f3287fff640 1 -- 172.21.15.71:0/3039775372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32881de130 con 0x7f3270004d40 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.415+0000 7f3287fff640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32880722f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3278007290 tx=0x7f3278007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.415+0000 7f3284ff9640 1 -- 172.21.15.71:0/3039775372 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327800f960 con 0x7f3270004d40 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.415+0000 7f3284ff9640 1 -- 172.21.15.71:0/3039775372 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3278018070 con 0x7f3270004d40 2024-04-16T15:41:28.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.415+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32880737c0 con 0x7f3270004d40 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.415+0000 7f3284ff9640 1 -- 172.21.15.71:0/3039775372 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327800fd60 con 0x7f3270004d40 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.415+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3288073200 con 0x7f3270004d40 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.416+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3288068d20 con 0x7f3270004d40 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.416+0000 7f3284ff9640 1 -- 172.21.15.71:0/3039775372 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f327801d460 con 0x7f3270004d40 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.417+0000 7f3284ff9640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f327406c920 0x7f327406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.417+0000 7f32877fe640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f327406c920 0x7f327406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.417+0000 7f3284ff9640 1 -- 172.21.15.71:0/3039775372 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f327808bdb0 con 0x7f3270004d40 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.417+0000 7f32877fe640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f327406c920 0x7f327406ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f327c007b60 tx=0x7f327c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:28.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.421+0000 7f3284ff9640 1 -- 172.21.15.71:0/3039775372 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32881de130 con 0x7f3270004d40 2024-04-16T15:41:28.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.638+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f328806bb50 con 0x7f3270004d40 2024-04-16T15:41:28.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.639+0000 7f3284ff9640 1 -- 172.21.15.71:0/3039775372 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f327805a050 con 0x7f3270004d40 2024-04-16T15:41:28.638 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:28.638 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6836/4129850104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4129850104},{"type":"v1","addr":"172.21.15.71:6836","nonce":4129850104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2014745378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2014745378},{"type":"v1","addr":"172.21.15.122:6834","nonce":2014745378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1432987242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1432987242},{"type":"v1","addr":"172.21.15.122:6835","nonce":1432987242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:41:22.867981+0000","modified":"2024-04-16T15:41:25.620418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5199},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5199":{"gid":5199,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6837/180716660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6835","nonce":180716660},{"type":"v1","addr":"172.21.15.71:6837","nonce":180716660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5199,"qdb_cluster":[5199]},"id":2}]} 2024-04-16T15:41:28.639 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 17 2024-04-16T15:41:28.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f327406c920 msgr2=0x7f327406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f327406c920 0x7f327406ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f327c007b60 tx=0x7f327c009040 comp rx=0 tx=0).stop 2024-04-16T15:41:28.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 msgr2=0x7f32880722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:28.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32880722f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3278007290 tx=0x7f3278007750 comp rx=0 tx=0).stop 2024-04-16T15:41:28.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 shutdown_connections 2024-04-16T15:41:28.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f327406c920 0x7f327406ede0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f32881dd770 0x7f32881ddbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32881de520 0x7f32880c09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 --2- 172.21.15.71:0/3039775372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3270004d40 0x7f32880722f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:28.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 >> 172.21.15.71:0/3039775372 conn(0x7f32880c3e10 msgr2=0x7f3288146300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:28.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 shutdown_connections 2024-04-16T15:41:28.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:28.641+0000 7f328e86a640 1 -- 172.21.15.71:0/3039775372 wait complete. 2024-04-16T15:41:28.655 INFO:tasks.cephfs_test_runner:That unsetting mds_join_fs will not cause failover if no high-affinity standby exists. ... ERROR 2024-04-16T15:41:28.655 INFO:tasks.cephfs_test_runner: 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner:ERROR: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner:That unsetting mds_join_fs will not cause failover if no high-affinity standby exists. 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks/cephfs/cephfs_test_case.py", line 180, in setUp 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner: self.fs.wait_for_daemons() 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks/cephfs/filesystem.py", line 1270, in wait_for_daemons 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner: if self.are_daemons_healthy(status=status, skip_max_mds_check=skip_max_mds_check): 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks/cephfs/filesystem.py", line 1075, in are_daemons_healthy 2024-04-16T15:41:28.656 INFO:tasks.cephfs_test_runner: mds.check_status() 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/daemon/state.py", line 43, in check_status 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: return self.proc.poll() 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 211, in poll 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner:Ran 1 test in 27.685s 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner: 2024-04-16T15:41:28.657 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-16T15:41:28.658 INFO:tasks.cephfs_test_runner:ERROR: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) 2024-04-16T15:41:28.658 INFO:tasks.cephfs_test_runner:That unsetting mds_join_fs will not cause failover if no high-affinity standby exists. 2024-04-16T15:41:28.658 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-16T15:41:28.658 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-16T15:41:28.658 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks/cephfs/cephfs_test_case.py", line 180, in setUp 2024-04-16T15:41:28.658 INFO:tasks.cephfs_test_runner: self.fs.wait_for_daemons() 2024-04-16T15:41:28.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks/cephfs/filesystem.py", line 1270, in wait_for_daemons 2024-04-16T15:41:28.659 INFO:tasks.cephfs_test_runner: if self.are_daemons_healthy(status=status, skip_max_mds_check=skip_max_mds_check): 2024-04-16T15:41:28.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks/cephfs/filesystem.py", line 1075, in are_daemons_healthy 2024-04-16T15:41:28.659 INFO:tasks.cephfs_test_runner: mds.check_status() 2024-04-16T15:41:28.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/daemon/state.py", line 43, in check_status 2024-04-16T15:41:28.659 INFO:tasks.cephfs_test_runner: return self.proc.poll() 2024-04-16T15:41:28.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 211, in poll 2024-04-16T15:41:28.660 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-16T15:41:28.660 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-16T15:41:28.660 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-16T15:41:28.660 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-04-16T15:41:28.660 INFO:tasks.cephfs_test_runner: 2024-04-16T15:41:28.661 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) 2024-04-16T15:41:28.926 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b1771acad0bb4f8fa02652d8cda49097 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_c3c87ed01bade92fb0ec360baa9bd9c82b8fde4f/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) 2024-04-16T15:41:28.928 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-16T15:41:28.944 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-16T15:41:28.967 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-16T15:41:28.967 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-16T15:41:28.967 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:41:28.987 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18654 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-04-16T15:41:28.987 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-16T15:41:28.987 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T15:41:29.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056329,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20509 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248808k,nr_inodes=812202,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-04-16T15:41:29.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:29.007 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-04-16T15:41:29.044 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-16T15:41:29.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:29.044 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T15:41:29.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:29.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-04-16T15:41:29.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:29.207 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-04-16T15:41:29.227 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-16T15:41:29.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:29.228 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-16T15:41:29.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T15:41:29.303 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-04-16T15:41:29.402 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-16T15:41:29.417 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-16T15:41:29.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:41:29.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.523+0000 7f49d2ee4640 1 Processor -- start 2024-04-16T15:41:29.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.523+0000 7f49d2ee4640 1 -- start start 2024-04-16T15:41:29.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d2ee4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc150f40 0x7f49cc0accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d2ee4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f49cc0ad210 0x7f49cc0aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d2ee4640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f49cc0ab950 0x7f49cc156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d2ee4640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f49cc153ba0 con 0x7f49cc0ab950 2024-04-16T15:41:29.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d2ee4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f49cc153900 con 0x7f49cc0ad210 2024-04-16T15:41:29.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d2ee4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc0ac4d0 con 0x7f49cc150f40 2024-04-16T15:41:29.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d1481640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f49cc0ab950 0x7f49cc156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d1481640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f49cc0ab950 0x7f49cc156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53662/0 (socket says 172.21.15.71:53662) 2024-04-16T15:41:29.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d1481640 1 -- 172.21.15.71:0/4255001477 learned_addr learned my addr 172.21.15.71:0/4255001477 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:29.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49d0c80640 1 --2- 172.21.15.71:0/4255001477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc150f40 0x7f49cc0accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.524+0000 7f49cbfff640 1 --2- 172.21.15.71:0/4255001477 >> v2:172.21.15.122:3300/0 conn(0x7f49cc0ad210 0x7f49cc0aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49d1481640 1 -- 172.21.15.71:0/4255001477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc150f40 msgr2=0x7f49cc0accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:29.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49d1481640 1 --2- 172.21.15.71:0/4255001477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc150f40 0x7f49cc0accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49d1481640 1 -- 172.21.15.71:0/4255001477 >> v2:172.21.15.122:3300/0 conn(0x7f49cc0ad210 msgr2=0x7f49cc0aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:29.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49d1481640 1 --2- 172.21.15.71:0/4255001477 >> v2:172.21.15.122:3300/0 conn(0x7f49cc0ad210 0x7f49cc0aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49d1481640 1 -- 172.21.15.71:0/4255001477 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc0ac7b0 con 0x7f49cc0ab950 2024-04-16T15:41:29.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49d1481640 1 --2- 172.21.15.71:0/4255001477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc0ab950 0x7f49cc156db0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0740e0 tx=0x7f49c402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a757090a68cbb560 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:29.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49cb7fe640 1 -- 172.21.15.71:0/4255001477 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4034070 con 0x7f49cc0ab950 2024-04-16T15:41:29.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49cb7fe640 1 -- 172.21.15.71:0/4255001477 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c4030860 con 0x7f49cc0ab950 2024-04-16T15:41:29.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.525+0000 7f49cb7fe640 1 -- 172.21.15.71:0/4255001477 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4030c30 con 0x7f49cc0ab950 2024-04-16T15:41:29.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4255001477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc0ab950 msgr2=0x7f49cc156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4255001477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc0ab950 0x7f49cc156db0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0740e0 tx=0x7f49c402b9a0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4255001477 shutdown_connections 2024-04-16T15:41:29.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4255001477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc0ab950 0x7f49cc156db0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4255001477 >> v2:172.21.15.122:3300/0 conn(0x7f49cc0ad210 0x7f49cc0aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4255001477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc150f40 0x7f49cc0accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4255001477 >> 172.21.15.71:0/4255001477 conn(0x7f49cc0c3e10 msgr2=0x7f49cc0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:29.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4255001477 shutdown_connections 2024-04-16T15:41:29.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.526+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4255001477 wait complete. 2024-04-16T15:41:29.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.527+0000 7f49d2ee4640 1 Processor -- start 2024-04-16T15:41:29.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.527+0000 7f49d2ee4640 1 -- start start 2024-04-16T15:41:29.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d2ee4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc0ab950 0x7f49cc1ed500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d2ee4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49cc150f40 0x7f49cc152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d2ee4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc149f00 0x7f49cc151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d2ee4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc06ff50 con 0x7f49cc149f00 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d2ee4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc06fdd0 con 0x7f49cc150f40 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d2ee4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc0c1840 con 0x7f49cc0ab950 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc149f00 0x7f49cc151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc149f00 0x7f49cc151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53674/0 (socket says 172.21.15.71:53674) 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 -- 172.21.15.71:0/4131940402 learned_addr learned my addr 172.21.15.71:0/4131940402 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d0c80640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc0ab950 0x7f49cc1ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49cbfff640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49cc150f40 0x7f49cc152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 -- 172.21.15.71:0/4131940402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc0ab950 msgr2=0x7f49cc1ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc0ab950 0x7f49cc1ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 -- 172.21.15.71:0/4131940402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49cc150f40 msgr2=0x7f49cc152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49cc150f40 0x7f49cc152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 -- 172.21.15.71:0/4131940402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc153000 con 0x7f49cc149f00 2024-04-16T15:41:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.528+0000 7f49d1481640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc149f00 0x7f49cc151da0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f49c402b9e0 tx=0x7f49c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.529+0000 7f49c97fa640 1 -- 172.21.15.71:0/4131940402 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4048070 con 0x7f49cc149f00 2024-04-16T15:41:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.529+0000 7f49c97fa640 1 -- 172.21.15.71:0/4131940402 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c4030660 con 0x7f49cc149f00 2024-04-16T15:41:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.529+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc1532e0 con 0x7f49cc149f00 2024-04-16T15:41:29.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.529+0000 7f49c97fa640 1 -- 172.21.15.71:0/4131940402 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4034040 con 0x7f49cc149f00 2024-04-16T15:41:29.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.529+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc1ecd00 con 0x7f49cc149f00 2024-04-16T15:41:29.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.530+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc068d20 con 0x7f49cc149f00 2024-04-16T15:41:29.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.530+0000 7f49c97fa640 1 -- 172.21.15.71:0/4131940402 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49c404b070 con 0x7f49cc149f00 2024-04-16T15:41:29.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.531+0000 7f49c97fa640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f49ac06c8f0 0x7f49ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.531+0000 7f49c97fa640 1 -- 172.21.15.71:0/4131940402 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f49c40add90 con 0x7f49cc149f00 2024-04-16T15:41:29.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.531+0000 7f49d0c80640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f49ac06c8f0 0x7f49ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.531+0000 7f49d0c80640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f49ac06c8f0 0x7f49ac06edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0039e0 tx=0x7f49bc009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:29.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.535+0000 7f49c97fa640 1 -- 172.21.15.71:0/4131940402 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49c407ba10 con 0x7f49cc149f00 2024-04-16T15:41:29.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.707+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49cc06d680 con 0x7f49ac06c8f0 2024-04-16T15:41:29.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.713+0000 7f49c97fa640 1 -- 172.21.15.71:0/4131940402 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f49cc06d680 con 0x7f49ac06c8f0 2024-04-16T15:41:29.714 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:29.715 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:41:29.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f49ac06c8f0 msgr2=0x7f49ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f49ac06c8f0 0x7f49ac06edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0039e0 tx=0x7f49bc009830 comp rx=0 tx=0).stop 2024-04-16T15:41:29.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc149f00 msgr2=0x7f49cc151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc149f00 0x7f49cc151da0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f49c402b9e0 tx=0x7f49c4004730 comp rx=0 tx=0).stop 2024-04-16T15:41:29.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 shutdown_connections 2024-04-16T15:41:29.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f49ac06c8f0 0x7f49ac06edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49cc149f00 0x7f49cc151da0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49cc150f40 0x7f49cc152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 --2- 172.21.15.71:0/4131940402 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f49cc0ab950 0x7f49cc1ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 >> 172.21.15.71:0/4131940402 conn(0x7f49cc0c3e10 msgr2=0x7f49cc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:29.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 shutdown_connections 2024-04-16T15:41:29.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.716+0000 7f49d2ee4640 1 -- 172.21.15.71:0/4131940402 wait complete. 2024-04-16T15:41:29.726 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-16T15:41:27.708043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220816,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753904,"statfs":{"total":773094113280,"available":772867997696,"internally_reserved":0,"allocated":4759552,"data_stored":2707140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001104"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530182+0000","last_change":"2024-04-16T15:41:23.523079+0000","last_active":"2024-04-16T15:41:24.530182+0000","last_peered":"2024-04-16T15:41:24.530182+0000","last_clean":"2024-04-16T15:41:24.530182+0000","last_became_active":"2024-04-16T15:41:23.522804+0000","last_became_peered":"2024-04-16T15:41:23.522804+0000","last_unstale":"2024-04-16T15:41:24.530182+0000","last_undegraded":"2024-04-16T15:41:24.530182+0000","last_fullsized":"2024-04-16T15:41:24.530182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530026+0000","last_change":"2024-04-16T15:41:23.518347+0000","last_active":"2024-04-16T15:41:24.530026+0000","last_peered":"2024-04-16T15:41:24.530026+0000","last_clean":"2024-04-16T15:41:24.530026+0000","last_became_active":"2024-04-16T15:41:23.517372+0000","last_became_peered":"2024-04-16T15:41:23.517372+0000","last_unstale":"2024-04-16T15:41:24.530026+0000","last_undegraded":"2024-04-16T15:41:24.530026+0000","last_fullsized":"2024-04-16T15:41:24.530026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.533697+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.533520+0000","last_became_peered":"2024-04-16T15:41:23.533520+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069345+0000","last_change":"2024-04-16T15:41:23.521655+0000","last_active":"2024-04-16T15:41:25.069345+0000","last_peered":"2024-04-16T15:41:25.069345+0000","last_clean":"2024-04-16T15:41:25.069345+0000","last_became_active":"2024-04-16T15:41:23.521419+0000","last_became_peered":"2024-04-16T15:41:23.521419+0000","last_unstale":"2024-04-16T15:41:25.069345+0000","last_undegraded":"2024-04-16T15:41:25.069345+0000","last_fullsized":"2024-04-16T15:41:25.069345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:48.607343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069423+0000","last_change":"2024-04-16T15:41:23.522246+0000","last_active":"2024-04-16T15:41:25.069423+0000","last_peered":"2024-04-16T15:41:25.069423+0000","last_clean":"2024-04-16T15:41:25.069423+0000","last_became_active":"2024-04-16T15:41:23.522009+0000","last_became_peered":"2024-04-16T15:41:23.522009+0000","last_unstale":"2024-04-16T15:41:25.069423+0000","last_undegraded":"2024-04-16T15:41:25.069423+0000","last_fullsized":"2024-04-16T15:41:25.069423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:33.728092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069091+0000","last_change":"2024-04-16T15:41:23.523004+0000","last_active":"2024-04-16T15:41:25.069091+0000","last_peered":"2024-04-16T15:41:25.069091+0000","last_clean":"2024-04-16T15:41:25.069091+0000","last_became_active":"2024-04-16T15:41:23.522742+0000","last_became_peered":"2024-04-16T15:41:23.522742+0000","last_unstale":"2024-04-16T15:41:25.069091+0000","last_undegraded":"2024-04-16T15:41:25.069091+0000","last_fullsized":"2024-04-16T15:41:25.069091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527561+0000","last_change":"2024-04-16T15:41:23.523305+0000","last_active":"2024-04-16T15:41:24.527561+0000","last_peered":"2024-04-16T15:41:24.527561+0000","last_clean":"2024-04-16T15:41:24.527561+0000","last_became_active":"2024-04-16T15:41:23.523022+0000","last_became_peered":"2024-04-16T15:41:23.523022+0000","last_unstale":"2024-04-16T15:41:24.527561+0000","last_undegraded":"2024-04-16T15:41:24.527561+0000","last_fullsized":"2024-04-16T15:41:24.527561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529666+0000","last_change":"2024-04-16T15:41:23.535446+0000","last_active":"2024-04-16T15:41:24.529666+0000","last_peered":"2024-04-16T15:41:24.529666+0000","last_clean":"2024-04-16T15:41:24.529666+0000","last_became_active":"2024-04-16T15:41:23.535156+0000","last_became_peered":"2024-04-16T15:41:23.535156+0000","last_unstale":"2024-04-16T15:41:24.529666+0000","last_undegraded":"2024-04-16T15:41:24.529666+0000","last_fullsized":"2024-04-16T15:41:24.529666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069784+0000","last_change":"2024-04-16T15:41:23.535019+0000","last_active":"2024-04-16T15:41:25.069784+0000","last_peered":"2024-04-16T15:41:25.069784+0000","last_clean":"2024-04-16T15:41:25.069784+0000","last_became_active":"2024-04-16T15:41:23.534726+0000","last_became_peered":"2024-04-16T15:41:23.534726+0000","last_unstale":"2024-04-16T15:41:25.069784+0000","last_undegraded":"2024-04-16T15:41:25.069784+0000","last_fullsized":"2024-04-16T15:41:25.069784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528288+0000","last_change":"2024-04-16T15:41:23.522709+0000","last_active":"2024-04-16T15:41:24.528288+0000","last_peered":"2024-04-16T15:41:24.528288+0000","last_clean":"2024-04-16T15:41:24.528288+0000","last_became_active":"2024-04-16T15:41:23.522437+0000","last_became_peered":"2024-04-16T15:41:23.522437+0000","last_unstale":"2024-04-16T15:41:24.528288+0000","last_undegraded":"2024-04-16T15:41:24.528288+0000","last_fullsized":"2024-04-16T15:41:24.528288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528764+0000","last_change":"2024-04-16T15:41:23.530620+0000","last_active":"2024-04-16T15:41:24.528764+0000","last_peered":"2024-04-16T15:41:24.528764+0000","last_clean":"2024-04-16T15:41:24.528764+0000","last_became_active":"2024-04-16T15:41:23.530320+0000","last_became_peered":"2024-04-16T15:41:23.530320+0000","last_unstale":"2024-04-16T15:41:24.528764+0000","last_undegraded":"2024-04-16T15:41:24.528764+0000","last_fullsized":"2024-04-16T15:41:24.528764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068927+0000","last_change":"2024-04-16T15:41:23.523419+0000","last_active":"2024-04-16T15:41:25.068927+0000","last_peered":"2024-04-16T15:41:25.068927+0000","last_clean":"2024-04-16T15:41:25.068927+0000","last_became_active":"2024-04-16T15:41:23.523081+0000","last_became_peered":"2024-04-16T15:41:23.523081+0000","last_unstale":"2024-04-16T15:41:25.068927+0000","last_undegraded":"2024-04-16T15:41:25.068927+0000","last_fullsized":"2024-04-16T15:41:25.068927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527828+0000","last_change":"2024-04-16T15:41:23.522227+0000","last_active":"2024-04-16T15:41:24.527828+0000","last_peered":"2024-04-16T15:41:24.527828+0000","last_clean":"2024-04-16T15:41:24.527828+0000","last_became_active":"2024-04-16T15:41:23.521531+0000","last_became_peered":"2024-04-16T15:41:23.521531+0000","last_unstale":"2024-04-16T15:41:24.527828+0000","last_undegraded":"2024-04-16T15:41:24.527828+0000","last_fullsized":"2024-04-16T15:41:24.527828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683427+0000","last_change":"2024-04-16T15:41:23.529710+0000","last_active":"2024-04-16T15:41:24.683427+0000","last_peered":"2024-04-16T15:41:24.683427+0000","last_clean":"2024-04-16T15:41:24.683427+0000","last_became_active":"2024-04-16T15:41:23.528557+0000","last_became_peered":"2024-04-16T15:41:23.528557+0000","last_unstale":"2024-04-16T15:41:24.683427+0000","last_undegraded":"2024-04-16T15:41:24.683427+0000","last_fullsized":"2024-04-16T15:41:24.683427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527689+0000","last_change":"2024-04-16T15:41:23.513641+0000","last_active":"2024-04-16T15:41:24.527689+0000","last_peered":"2024-04-16T15:41:24.527689+0000","last_clean":"2024-04-16T15:41:24.527689+0000","last_became_active":"2024-04-16T15:41:23.513340+0000","last_became_peered":"2024-04-16T15:41:23.513340+0000","last_unstale":"2024-04-16T15:41:24.527689+0000","last_undegraded":"2024-04-16T15:41:24.527689+0000","last_fullsized":"2024-04-16T15:41:24.527689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527345+0000","last_change":"2024-04-16T15:41:23.535782+0000","last_active":"2024-04-16T15:41:24.527345+0000","last_peered":"2024-04-16T15:41:24.527345+0000","last_clean":"2024-04-16T15:41:24.527345+0000","last_became_active":"2024-04-16T15:41:23.535549+0000","last_became_peered":"2024-04-16T15:41:23.535549+0000","last_unstale":"2024-04-16T15:41:24.527345+0000","last_undegraded":"2024-04-16T15:41:24.527345+0000","last_fullsized":"2024-04-16T15:41:24.527345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528422+0000","last_change":"2024-04-16T15:41:23.536402+0000","last_active":"2024-04-16T15:41:24.528422+0000","last_peered":"2024-04-16T15:41:24.528422+0000","last_clean":"2024-04-16T15:41:24.528422+0000","last_became_active":"2024-04-16T15:41:23.536009+0000","last_became_peered":"2024-04-16T15:41:23.536009+0000","last_unstale":"2024-04-16T15:41:24.528422+0000","last_undegraded":"2024-04-16T15:41:24.528422+0000","last_fullsized":"2024-04-16T15:41:24.528422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068843+0000","last_change":"2024-04-16T15:41:23.523701+0000","last_active":"2024-04-16T15:41:25.068843+0000","last_peered":"2024-04-16T15:41:25.068843+0000","last_clean":"2024-04-16T15:41:25.068843+0000","last_became_active":"2024-04-16T15:41:23.523442+0000","last_became_peered":"2024-04-16T15:41:23.523442+0000","last_unstale":"2024-04-16T15:41:25.068843+0000","last_undegraded":"2024-04-16T15:41:25.068843+0000","last_fullsized":"2024-04-16T15:41:25.068843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070293+0000","last_change":"2024-04-16T15:41:23.519553+0000","last_active":"2024-04-16T15:41:25.070293+0000","last_peered":"2024-04-16T15:41:25.070293+0000","last_clean":"2024-04-16T15:41:25.070293+0000","last_became_active":"2024-04-16T15:41:23.519258+0000","last_became_peered":"2024-04-16T15:41:23.519258+0000","last_unstale":"2024-04-16T15:41:25.070293+0000","last_undegraded":"2024-04-16T15:41:25.070293+0000","last_fullsized":"2024-04-16T15:41:25.070293+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:51.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068696+0000","last_change":"2024-04-16T15:41:23.522212+0000","last_active":"2024-04-16T15:41:25.068696+0000","last_peered":"2024-04-16T15:41:25.068696+0000","last_clean":"2024-04-16T15:41:25.068696+0000","last_became_active":"2024-04-16T15:41:23.521934+0000","last_became_peered":"2024-04-16T15:41:23.521934+0000","last_unstale":"2024-04-16T15:41:25.068696+0000","last_undegraded":"2024-04-16T15:41:25.068696+0000","last_fullsized":"2024-04-16T15:41:25.068696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:03.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527973+0000","last_change":"2024-04-16T15:41:23.520977+0000","last_active":"2024-04-16T15:41:24.527973+0000","last_peered":"2024-04-16T15:41:24.527973+0000","last_clean":"2024-04-16T15:41:24.527973+0000","last_became_active":"2024-04-16T15:41:23.520677+0000","last_became_peered":"2024-04-16T15:41:23.520677+0000","last_unstale":"2024-04-16T15:41:24.527973+0000","last_undegraded":"2024-04-16T15:41:24.527973+0000","last_fullsized":"2024-04-16T15:41:24.527973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683729+0000","last_change":"2024-04-16T15:41:23.527968+0000","last_active":"2024-04-16T15:41:24.683729+0000","last_peered":"2024-04-16T15:41:24.683729+0000","last_clean":"2024-04-16T15:41:24.683729+0000","last_became_active":"2024-04-16T15:41:23.527690+0000","last_became_peered":"2024-04-16T15:41:23.527690+0000","last_unstale":"2024-04-16T15:41:24.683729+0000","last_undegraded":"2024-04-16T15:41:24.683729+0000","last_fullsized":"2024-04-16T15:41:24.683729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527416+0000","last_change":"2024-04-16T15:41:23.527691+0000","last_active":"2024-04-16T15:41:24.527416+0000","last_peered":"2024-04-16T15:41:24.527416+0000","last_clean":"2024-04-16T15:41:24.527416+0000","last_became_active":"2024-04-16T15:41:23.527372+0000","last_became_peered":"2024-04-16T15:41:23.527372+0000","last_unstale":"2024-04-16T15:41:24.527416+0000","last_undegraded":"2024-04-16T15:41:24.527416+0000","last_fullsized":"2024-04-16T15:41:24.527416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069312+0000","last_change":"2024-04-16T15:41:23.525188+0000","last_active":"2024-04-16T15:41:25.069312+0000","last_peered":"2024-04-16T15:41:25.069312+0000","last_clean":"2024-04-16T15:41:25.069312+0000","last_became_active":"2024-04-16T15:41:23.524939+0000","last_became_peered":"2024-04-16T15:41:23.524939+0000","last_unstale":"2024-04-16T15:41:25.069312+0000","last_undegraded":"2024-04-16T15:41:25.069312+0000","last_fullsized":"2024-04-16T15:41:25.069312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:53.692430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068745+0000","last_change":"2024-04-16T15:41:23.526266+0000","last_active":"2024-04-16T15:41:25.068745+0000","last_peered":"2024-04-16T15:41:25.068745+0000","last_clean":"2024-04-16T15:41:25.068745+0000","last_became_active":"2024-04-16T15:41:23.525999+0000","last_became_peered":"2024-04-16T15:41:23.525999+0000","last_unstale":"2024-04-16T15:41:25.068745+0000","last_undegraded":"2024-04-16T15:41:25.068745+0000","last_fullsized":"2024-04-16T15:41:25.068745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:38.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069155+0000","last_change":"2024-04-16T15:41:23.535525+0000","last_active":"2024-04-16T15:41:25.069155+0000","last_peered":"2024-04-16T15:41:25.069155+0000","last_clean":"2024-04-16T15:41:25.069155+0000","last_became_active":"2024-04-16T15:41:23.535192+0000","last_became_peered":"2024-04-16T15:41:23.535192+0000","last_unstale":"2024-04-16T15:41:25.069155+0000","last_undegraded":"2024-04-16T15:41:25.069155+0000","last_fullsized":"2024-04-16T15:41:25.069155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068809+0000","last_change":"2024-04-16T15:41:23.527164+0000","last_active":"2024-04-16T15:41:25.068809+0000","last_peered":"2024-04-16T15:41:25.068809+0000","last_clean":"2024-04-16T15:41:25.068809+0000","last_became_active":"2024-04-16T15:41:23.526765+0000","last_became_peered":"2024-04-16T15:41:23.526765+0000","last_unstale":"2024-04-16T15:41:25.068809+0000","last_undegraded":"2024-04-16T15:41:25.068809+0000","last_fullsized":"2024-04-16T15:41:25.068809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528932+0000","last_change":"2024-04-16T15:41:23.529704+0000","last_active":"2024-04-16T15:41:24.528932+0000","last_peered":"2024-04-16T15:41:24.528932+0000","last_clean":"2024-04-16T15:41:24.528932+0000","last_became_active":"2024-04-16T15:41:23.525283+0000","last_became_peered":"2024-04-16T15:41:23.525283+0000","last_unstale":"2024-04-16T15:41:24.528932+0000","last_undegraded":"2024-04-16T15:41:24.528932+0000","last_fullsized":"2024-04-16T15:41:24.528932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068718+0000","last_change":"2024-04-16T15:41:23.523057+0000","last_active":"2024-04-16T15:41:25.068718+0000","last_peered":"2024-04-16T15:41:25.068718+0000","last_clean":"2024-04-16T15:41:25.068718+0000","last_became_active":"2024-04-16T15:41:23.522762+0000","last_became_peered":"2024-04-16T15:41:23.522762+0000","last_unstale":"2024-04-16T15:41:25.068718+0000","last_undegraded":"2024-04-16T15:41:25.068718+0000","last_fullsized":"2024-04-16T15:41:25.068718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:57.277773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068960+0000","last_change":"2024-04-16T15:41:23.522445+0000","last_active":"2024-04-16T15:41:25.068960+0000","last_peered":"2024-04-16T15:41:25.068960+0000","last_clean":"2024-04-16T15:41:25.068960+0000","last_became_active":"2024-04-16T15:41:23.522179+0000","last_became_peered":"2024-04-16T15:41:23.522179+0000","last_unstale":"2024-04-16T15:41:25.068960+0000","last_undegraded":"2024-04-16T15:41:25.068960+0000","last_fullsized":"2024-04-16T15:41:25.068960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527557+0000","last_change":"2024-04-16T15:41:23.528582+0000","last_active":"2024-04-16T15:41:24.527557+0000","last_peered":"2024-04-16T15:41:24.527557+0000","last_clean":"2024-04-16T15:41:24.527557+0000","last_became_active":"2024-04-16T15:41:23.528304+0000","last_became_peered":"2024-04-16T15:41:23.528304+0000","last_unstale":"2024-04-16T15:41:24.527557+0000","last_undegraded":"2024-04-16T15:41:24.527557+0000","last_fullsized":"2024-04-16T15:41:24.527557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:58:58.124408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069657+0000","last_change":"2024-04-16T15:41:23.519347+0000","last_active":"2024-04-16T15:41:25.069657+0000","last_peered":"2024-04-16T15:41:25.069657+0000","last_clean":"2024-04-16T15:41:25.069657+0000","last_became_active":"2024-04-16T15:41:23.519092+0000","last_became_peered":"2024-04-16T15:41:23.519092+0000","last_unstale":"2024-04-16T15:41:25.069657+0000","last_undegraded":"2024-04-16T15:41:25.069657+0000","last_fullsized":"2024-04-16T15:41:25.069657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527966+0000","last_change":"2024-04-16T15:41:23.536251+0000","last_active":"2024-04-16T15:41:24.527966+0000","last_peered":"2024-04-16T15:41:24.527966+0000","last_clean":"2024-04-16T15:41:24.527966+0000","last_became_active":"2024-04-16T15:41:23.535963+0000","last_became_peered":"2024-04-16T15:41:23.535963+0000","last_unstale":"2024-04-16T15:41:24.527966+0000","last_undegraded":"2024-04-16T15:41:24.527966+0000","last_fullsized":"2024-04-16T15:41:24.527966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068769+0000","last_change":"2024-04-16T15:41:22.522796+0000","last_active":"2024-04-16T15:41:25.068769+0000","last_peered":"2024-04-16T15:41:25.068769+0000","last_clean":"2024-04-16T15:41:25.068769+0000","last_became_active":"2024-04-16T15:41:22.522542+0000","last_became_peered":"2024-04-16T15:41:22.522542+0000","last_unstale":"2024-04-16T15:41:25.068769+0000","last_undegraded":"2024-04-16T15:41:25.068769+0000","last_fullsized":"2024-04-16T15:41:25.068769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069565+0000","last_change":"2024-04-16T15:41:23.536192+0000","last_active":"2024-04-16T15:41:25.069565+0000","last_peered":"2024-04-16T15:41:25.069565+0000","last_clean":"2024-04-16T15:41:25.069565+0000","last_became_active":"2024-04-16T15:41:23.535920+0000","last_became_peered":"2024-04-16T15:41:23.535920+0000","last_unstale":"2024-04-16T15:41:25.069565+0000","last_undegraded":"2024-04-16T15:41:25.069565+0000","last_fullsized":"2024-04-16T15:41:25.069565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069482+0000","last_change":"2024-04-16T15:41:22.523464+0000","last_active":"2024-04-16T15:41:25.069482+0000","last_peered":"2024-04-16T15:41:25.069482+0000","last_clean":"2024-04-16T15:41:25.069482+0000","last_became_active":"2024-04-16T15:41:22.523128+0000","last_became_peered":"2024-04-16T15:41:22.523128+0000","last_unstale":"2024-04-16T15:41:25.069482+0000","last_undegraded":"2024-04-16T15:41:25.069482+0000","last_fullsized":"2024-04-16T15:41:25.069482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683674+0000","last_change":"2024-04-16T15:41:23.530373+0000","last_active":"2024-04-16T15:41:24.683674+0000","last_peered":"2024-04-16T15:41:24.683674+0000","last_clean":"2024-04-16T15:41:24.683674+0000","last_became_active":"2024-04-16T15:41:23.530115+0000","last_became_peered":"2024-04-16T15:41:23.530115+0000","last_unstale":"2024-04-16T15:41:24.683674+0000","last_undegraded":"2024-04-16T15:41:24.683674+0000","last_fullsized":"2024-04-16T15:41:24.683674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069538+0000","last_change":"2024-04-16T15:41:22.521948+0000","last_active":"2024-04-16T15:41:25.069538+0000","last_peered":"2024-04-16T15:41:25.069538+0000","last_clean":"2024-04-16T15:41:25.069538+0000","last_became_active":"2024-04-16T15:41:22.521643+0000","last_became_peered":"2024-04-16T15:41:22.521643+0000","last_unstale":"2024-04-16T15:41:25.069538+0000","last_undegraded":"2024-04-16T15:41:25.069538+0000","last_fullsized":"2024-04-16T15:41:25.069538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530028+0000","last_change":"2024-04-16T15:41:23.528874+0000","last_active":"2024-04-16T15:41:24.530028+0000","last_peered":"2024-04-16T15:41:24.530028+0000","last_clean":"2024-04-16T15:41:24.530028+0000","last_became_active":"2024-04-16T15:41:23.528569+0000","last_became_peered":"2024-04-16T15:41:23.528569+0000","last_unstale":"2024-04-16T15:41:24.530028+0000","last_undegraded":"2024-04-16T15:41:24.530028+0000","last_fullsized":"2024-04-16T15:41:24.530028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683536+0000","last_change":"2024-04-16T15:41:22.525207+0000","last_active":"2024-04-16T15:41:24.683536+0000","last_peered":"2024-04-16T15:41:24.683536+0000","last_clean":"2024-04-16T15:41:24.683536+0000","last_became_active":"2024-04-16T15:41:22.524864+0000","last_became_peered":"2024-04-16T15:41:22.524864+0000","last_unstale":"2024-04-16T15:41:24.683536+0000","last_undegraded":"2024-04-16T15:41:24.683536+0000","last_fullsized":"2024-04-16T15:41:24.683536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068873+0000","last_change":"2024-04-16T15:41:23.523861+0000","last_active":"2024-04-16T15:41:25.068873+0000","last_peered":"2024-04-16T15:41:25.068873+0000","last_clean":"2024-04-16T15:41:25.068873+0000","last_became_active":"2024-04-16T15:41:23.523587+0000","last_became_peered":"2024-04-16T15:41:23.523587+0000","last_unstale":"2024-04-16T15:41:25.068873+0000","last_undegraded":"2024-04-16T15:41:25.068873+0000","last_fullsized":"2024-04-16T15:41:25.068873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:22.526265+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:22.525849+0000","last_became_peered":"2024-04-16T15:41:22.525849+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:02.400517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528806+0000","last_change":"2024-04-16T15:41:23.521523+0000","last_active":"2024-04-16T15:41:24.528806+0000","last_peered":"2024-04-16T15:41:24.528806+0000","last_clean":"2024-04-16T15:41:24.528806+0000","last_became_active":"2024-04-16T15:41:23.520768+0000","last_became_peered":"2024-04-16T15:41:23.520768+0000","last_unstale":"2024-04-16T15:41:24.528806+0000","last_undegraded":"2024-04-16T15:41:24.528806+0000","last_fullsized":"2024-04-16T15:41:24.528806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527590+0000","last_change":"2024-04-16T15:41:22.528402+0000","last_active":"2024-04-16T15:41:24.527590+0000","last_peered":"2024-04-16T15:41:24.527590+0000","last_clean":"2024-04-16T15:41:24.527590+0000","last_became_active":"2024-04-16T15:41:22.527999+0000","last_became_peered":"2024-04-16T15:41:22.527999+0000","last_unstale":"2024-04-16T15:41:24.527590+0000","last_undegraded":"2024-04-16T15:41:24.527590+0000","last_fullsized":"2024-04-16T15:41:24.527590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683644+0000","last_change":"2024-04-16T15:41:23.535673+0000","last_active":"2024-04-16T15:41:24.683644+0000","last_peered":"2024-04-16T15:41:24.683644+0000","last_clean":"2024-04-16T15:41:24.683644+0000","last_became_active":"2024-04-16T15:41:23.535387+0000","last_became_peered":"2024-04-16T15:41:23.535387+0000","last_unstale":"2024-04-16T15:41:24.683644+0000","last_undegraded":"2024-04-16T15:41:24.683644+0000","last_fullsized":"2024-04-16T15:41:24.683644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070111+0000","last_change":"2024-04-16T15:41:22.526184+0000","last_active":"2024-04-16T15:41:25.070111+0000","last_peered":"2024-04-16T15:41:25.070111+0000","last_clean":"2024-04-16T15:41:25.070111+0000","last_became_active":"2024-04-16T15:41:22.525819+0000","last_became_peered":"2024-04-16T15:41:22.525819+0000","last_unstale":"2024-04-16T15:41:25.070111+0000","last_undegraded":"2024-04-16T15:41:25.070111+0000","last_fullsized":"2024-04-16T15:41:25.070111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530001+0000","last_change":"2024-04-16T15:41:23.522193+0000","last_active":"2024-04-16T15:41:24.530001+0000","last_peered":"2024-04-16T15:41:24.530001+0000","last_clean":"2024-04-16T15:41:24.530001+0000","last_became_active":"2024-04-16T15:41:23.521900+0000","last_became_peered":"2024-04-16T15:41:23.521900+0000","last_unstale":"2024-04-16T15:41:24.530001+0000","last_undegraded":"2024-04-16T15:41:24.530001+0000","last_fullsized":"2024-04-16T15:41:24.530001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.521519+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.521234+0000","last_became_peered":"2024-04-16T15:41:22.521234+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.521070+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.520819+0000","last_became_peered":"2024-04-16T15:41:23.520819+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:03:41.341706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530092+0000","last_change":"2024-04-16T15:41:22.519163+0000","last_active":"2024-04-16T15:41:24.530092+0000","last_peered":"2024-04-16T15:41:24.530092+0000","last_clean":"2024-04-16T15:41:24.530092+0000","last_became_active":"2024-04-16T15:41:22.518905+0000","last_became_peered":"2024-04-16T15:41:22.518905+0000","last_unstale":"2024-04-16T15:41:24.530092+0000","last_undegraded":"2024-04-16T15:41:24.530092+0000","last_fullsized":"2024-04-16T15:41:24.530092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:03.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528974+0000","last_change":"2024-04-16T15:41:23.528995+0000","last_active":"2024-04-16T15:41:24.528974+0000","last_peered":"2024-04-16T15:41:24.528974+0000","last_clean":"2024-04-16T15:41:24.528974+0000","last_became_active":"2024-04-16T15:41:23.526288+0000","last_became_peered":"2024-04-16T15:41:23.526288+0000","last_unstale":"2024-04-16T15:41:24.528974+0000","last_undegraded":"2024-04-16T15:41:24.528974+0000","last_fullsized":"2024-04-16T15:41:24.528974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528459+0000","last_change":"2024-04-16T15:41:22.525433+0000","last_active":"2024-04-16T15:41:24.528459+0000","last_peered":"2024-04-16T15:41:24.528459+0000","last_clean":"2024-04-16T15:41:24.528459+0000","last_became_active":"2024-04-16T15:41:22.524567+0000","last_became_peered":"2024-04-16T15:41:22.524567+0000","last_unstale":"2024-04-16T15:41:24.528459+0000","last_undegraded":"2024-04-16T15:41:24.528459+0000","last_fullsized":"2024-04-16T15:41:24.528459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:56.272300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069430+0000","last_change":"2024-04-16T15:41:23.522367+0000","last_active":"2024-04-16T15:41:25.069430+0000","last_peered":"2024-04-16T15:41:25.069430+0000","last_clean":"2024-04-16T15:41:25.069430+0000","last_became_active":"2024-04-16T15:41:23.521984+0000","last_became_peered":"2024-04-16T15:41:23.521984+0000","last_unstale":"2024-04-16T15:41:25.069430+0000","last_undegraded":"2024-04-16T15:41:25.069430+0000","last_fullsized":"2024-04-16T15:41:25.069430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528812+0000","last_change":"2024-04-16T15:41:22.523086+0000","last_active":"2024-04-16T15:41:24.528812+0000","last_peered":"2024-04-16T15:41:24.528812+0000","last_clean":"2024-04-16T15:41:24.528812+0000","last_became_active":"2024-04-16T15:41:22.522759+0000","last_became_peered":"2024-04-16T15:41:22.522759+0000","last_unstale":"2024-04-16T15:41:24.528812+0000","last_undegraded":"2024-04-16T15:41:24.528812+0000","last_fullsized":"2024-04-16T15:41:24.528812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683778+0000","last_change":"2024-04-16T15:41:23.529819+0000","last_active":"2024-04-16T15:41:24.683778+0000","last_peered":"2024-04-16T15:41:24.683778+0000","last_clean":"2024-04-16T15:41:24.683778+0000","last_became_active":"2024-04-16T15:41:23.529543+0000","last_became_peered":"2024-04-16T15:41:23.529543+0000","last_unstale":"2024-04-16T15:41:24.683778+0000","last_undegraded":"2024-04-16T15:41:24.683778+0000","last_fullsized":"2024-04-16T15:41:24.683778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:22.522411+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:22.522130+0000","last_became_peered":"2024-04-16T15:41:22.522130+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:41.323112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069849+0000","last_change":"2024-04-16T15:41:23.535122+0000","last_active":"2024-04-16T15:41:25.069849+0000","last_peered":"2024-04-16T15:41:25.069849+0000","last_clean":"2024-04-16T15:41:25.069849+0000","last_became_active":"2024-04-16T15:41:23.534825+0000","last_became_peered":"2024-04-16T15:41:23.534825+0000","last_unstale":"2024-04-16T15:41:25.069849+0000","last_undegraded":"2024-04-16T15:41:25.069849+0000","last_fullsized":"2024-04-16T15:41:25.069849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530105+0000","last_change":"2024-04-16T15:41:22.520252+0000","last_active":"2024-04-16T15:41:24.530105+0000","last_peered":"2024-04-16T15:41:24.530105+0000","last_clean":"2024-04-16T15:41:24.530105+0000","last_became_active":"2024-04-16T15:41:22.519829+0000","last_became_peered":"2024-04-16T15:41:22.519829+0000","last_unstale":"2024-04-16T15:41:24.530105+0000","last_undegraded":"2024-04-16T15:41:24.530105+0000","last_fullsized":"2024-04-16T15:41:24.530105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529732+0000","last_change":"2024-04-16T15:41:23.522432+0000","last_active":"2024-04-16T15:41:24.529732+0000","last_peered":"2024-04-16T15:41:24.529732+0000","last_clean":"2024-04-16T15:41:24.529732+0000","last_became_active":"2024-04-16T15:41:23.522159+0000","last_became_peered":"2024-04-16T15:41:23.522159+0000","last_unstale":"2024-04-16T15:41:24.529732+0000","last_undegraded":"2024-04-16T15:41:24.529732+0000","last_fullsized":"2024-04-16T15:41:24.529732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069145+0000","last_change":"2024-04-16T15:41:22.527064+0000","last_active":"2024-04-16T15:41:25.069145+0000","last_peered":"2024-04-16T15:41:25.069145+0000","last_clean":"2024-04-16T15:41:25.069145+0000","last_became_active":"2024-04-16T15:41:22.525811+0000","last_became_peered":"2024-04-16T15:41:22.525811+0000","last_unstale":"2024-04-16T15:41:25.069145+0000","last_undegraded":"2024-04-16T15:41:25.069145+0000","last_fullsized":"2024-04-16T15:41:25.069145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527532+0000","last_change":"2024-04-16T15:41:23.527859+0000","last_active":"2024-04-16T15:41:24.527532+0000","last_peered":"2024-04-16T15:41:24.527532+0000","last_clean":"2024-04-16T15:41:24.527532+0000","last_became_active":"2024-04-16T15:41:23.527295+0000","last_became_peered":"2024-04-16T15:41:23.527295+0000","last_unstale":"2024-04-16T15:41:24.527532+0000","last_undegraded":"2024-04-16T15:41:24.527532+0000","last_fullsized":"2024-04-16T15:41:24.527532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:00:18.934742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068514+0000","last_change":"2024-04-16T15:41:22.524475+0000","last_active":"2024-04-16T15:41:25.068514+0000","last_peered":"2024-04-16T15:41:25.068514+0000","last_clean":"2024-04-16T15:41:25.068514+0000","last_became_active":"2024-04-16T15:41:22.523303+0000","last_became_peered":"2024-04-16T15:41:22.523303+0000","last_unstale":"2024-04-16T15:41:25.068514+0000","last_undegraded":"2024-04-16T15:41:25.068514+0000","last_fullsized":"2024-04-16T15:41:25.068514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:23.523183+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:23.522845+0000","last_became_peered":"2024-04-16T15:41:23.522845+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527394+0000","last_change":"2024-04-16T15:41:22.522941+0000","last_active":"2024-04-16T15:41:24.527394+0000","last_peered":"2024-04-16T15:41:24.527394+0000","last_clean":"2024-04-16T15:41:24.527394+0000","last_became_active":"2024-04-16T15:41:22.522633+0000","last_became_peered":"2024-04-16T15:41:22.522633+0000","last_unstale":"2024-04-16T15:41:24.527394+0000","last_undegraded":"2024-04-16T15:41:24.527394+0000","last_fullsized":"2024-04-16T15:41:24.527394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529991+0000","last_change":"2024-04-16T15:41:23.528140+0000","last_active":"2024-04-16T15:41:24.529991+0000","last_peered":"2024-04-16T15:41:24.529991+0000","last_clean":"2024-04-16T15:41:24.529991+0000","last_became_active":"2024-04-16T15:41:23.527841+0000","last_became_peered":"2024-04-16T15:41:23.527841+0000","last_unstale":"2024-04-16T15:41:24.529991+0000","last_undegraded":"2024-04-16T15:41:24.529991+0000","last_fullsized":"2024-04-16T15:41:24.529991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528233+0000","last_change":"2024-04-16T15:41:22.526961+0000","last_active":"2024-04-16T15:41:24.528233+0000","last_peered":"2024-04-16T15:41:24.528233+0000","last_clean":"2024-04-16T15:41:24.528233+0000","last_became_active":"2024-04-16T15:41:22.526140+0000","last_became_peered":"2024-04-16T15:41:22.526140+0000","last_unstale":"2024-04-16T15:41:24.528233+0000","last_undegraded":"2024-04-16T15:41:24.528233+0000","last_fullsized":"2024-04-16T15:41:24.528233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528839+0000","last_change":"2024-04-16T15:41:23.521298+0000","last_active":"2024-04-16T15:41:24.528839+0000","last_peered":"2024-04-16T15:41:24.528839+0000","last_clean":"2024-04-16T15:41:24.528839+0000","last_became_active":"2024-04-16T15:41:23.521041+0000","last_became_peered":"2024-04-16T15:41:23.521041+0000","last_unstale":"2024-04-16T15:41:24.528839+0000","last_undegraded":"2024-04-16T15:41:24.528839+0000","last_fullsized":"2024-04-16T15:41:24.528839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683715+0000","last_change":"2024-04-16T15:41:22.529046+0000","last_active":"2024-04-16T15:41:24.683715+0000","last_peered":"2024-04-16T15:41:24.683715+0000","last_clean":"2024-04-16T15:41:24.683715+0000","last_became_active":"2024-04-16T15:41:22.528745+0000","last_became_peered":"2024-04-16T15:41:22.528745+0000","last_unstale":"2024-04-16T15:41:24.683715+0000","last_undegraded":"2024-04-16T15:41:24.683715+0000","last_fullsized":"2024-04-16T15:41:24.683715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683671+0000","last_change":"2024-04-16T15:41:23.522758+0000","last_active":"2024-04-16T15:41:24.683671+0000","last_peered":"2024-04-16T15:41:24.683671+0000","last_clean":"2024-04-16T15:41:24.683671+0000","last_became_active":"2024-04-16T15:41:23.521964+0000","last_became_peered":"2024-04-16T15:41:23.521964+0000","last_unstale":"2024-04-16T15:41:24.683671+0000","last_undegraded":"2024-04-16T15:41:24.683671+0000","last_fullsized":"2024-04-16T15:41:24.683671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070256+0000","last_change":"2024-04-16T15:41:22.523228+0000","last_active":"2024-04-16T15:41:25.070256+0000","last_peered":"2024-04-16T15:41:25.070256+0000","last_clean":"2024-04-16T15:41:25.070256+0000","last_became_active":"2024-04-16T15:41:22.522888+0000","last_became_peered":"2024-04-16T15:41:22.522888+0000","last_unstale":"2024-04-16T15:41:25.070256+0000","last_undegraded":"2024-04-16T15:41:25.070256+0000","last_fullsized":"2024-04-16T15:41:25.070256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:59.195492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069536+0000","last_change":"2024-04-16T15:41:23.525797+0000","last_active":"2024-04-16T15:41:25.069536+0000","last_peered":"2024-04-16T15:41:25.069536+0000","last_clean":"2024-04-16T15:41:25.069536+0000","last_became_active":"2024-04-16T15:41:23.525537+0000","last_became_peered":"2024-04-16T15:41:23.525537+0000","last_unstale":"2024-04-16T15:41:25.069536+0000","last_undegraded":"2024-04-16T15:41:25.069536+0000","last_fullsized":"2024-04-16T15:41:25.069536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:06.161198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068884+0000","last_change":"2024-04-16T15:41:22.521885+0000","last_active":"2024-04-16T15:41:25.068884+0000","last_peered":"2024-04-16T15:41:25.068884+0000","last_clean":"2024-04-16T15:41:25.068884+0000","last_became_active":"2024-04-16T15:41:22.521574+0000","last_became_peered":"2024-04-16T15:41:22.521574+0000","last_unstale":"2024-04-16T15:41:25.068884+0000","last_undegraded":"2024-04-16T15:41:25.068884+0000","last_fullsized":"2024-04-16T15:41:25.068884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527576+0000","last_change":"2024-04-16T15:41:23.529208+0000","last_active":"2024-04-16T15:41:24.527576+0000","last_peered":"2024-04-16T15:41:24.527576+0000","last_clean":"2024-04-16T15:41:24.527576+0000","last_became_active":"2024-04-16T15:41:23.528946+0000","last_became_peered":"2024-04-16T15:41:23.528946+0000","last_unstale":"2024-04-16T15:41:24.527576+0000","last_undegraded":"2024-04-16T15:41:24.527576+0000","last_fullsized":"2024-04-16T15:41:24.527576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:59:03.633363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069471+0000","last_change":"2024-04-16T15:41:22.525515+0000","last_active":"2024-04-16T15:41:25.069471+0000","last_peered":"2024-04-16T15:41:25.069471+0000","last_clean":"2024-04-16T15:41:25.069471+0000","last_became_active":"2024-04-16T15:41:22.525228+0000","last_became_peered":"2024-04-16T15:41:22.525228+0000","last_unstale":"2024-04-16T15:41:25.069471+0000","last_undegraded":"2024-04-16T15:41:25.069471+0000","last_fullsized":"2024-04-16T15:41:25.069471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:55:38.421964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068802+0000","last_change":"2024-04-16T15:41:23.522196+0000","last_active":"2024-04-16T15:41:25.068802+0000","last_peered":"2024-04-16T15:41:25.068802+0000","last_clean":"2024-04-16T15:41:25.068802+0000","last_became_active":"2024-04-16T15:41:23.521931+0000","last_became_peered":"2024-04-16T15:41:23.521931+0000","last_unstale":"2024-04-16T15:41:25.068802+0000","last_undegraded":"2024-04-16T15:41:25.068802+0000","last_fullsized":"2024-04-16T15:41:25.068802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069160+0000","last_change":"2024-04-16T15:41:22.523743+0000","last_active":"2024-04-16T15:41:25.069160+0000","last_peered":"2024-04-16T15:41:25.069160+0000","last_clean":"2024-04-16T15:41:25.069160+0000","last_became_active":"2024-04-16T15:41:22.523348+0000","last_became_peered":"2024-04-16T15:41:22.523348+0000","last_unstale":"2024-04-16T15:41:25.069160+0000","last_undegraded":"2024-04-16T15:41:25.069160+0000","last_fullsized":"2024-04-16T15:41:25.069160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:48.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528766+0000","last_change":"2024-04-16T15:41:23.524838+0000","last_active":"2024-04-16T15:41:24.528766+0000","last_peered":"2024-04-16T15:41:24.528766+0000","last_clean":"2024-04-16T15:41:24.528766+0000","last_became_active":"2024-04-16T15:41:23.524540+0000","last_became_peered":"2024-04-16T15:41:23.524540+0000","last_unstale":"2024-04-16T15:41:24.528766+0000","last_undegraded":"2024-04-16T15:41:24.528766+0000","last_fullsized":"2024-04-16T15:41:24.528766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527606+0000","last_change":"2024-04-16T15:41:22.522239+0000","last_active":"2024-04-16T15:41:24.527606+0000","last_peered":"2024-04-16T15:41:24.527606+0000","last_clean":"2024-04-16T15:41:24.527606+0000","last_became_active":"2024-04-16T15:41:22.521860+0000","last_became_peered":"2024-04-16T15:41:22.521860+0000","last_unstale":"2024-04-16T15:41:24.527606+0000","last_undegraded":"2024-04-16T15:41:24.527606+0000","last_fullsized":"2024-04-16T15:41:24.527606+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530187+0000","last_change":"2024-04-16T15:41:23.522134+0000","last_active":"2024-04-16T15:41:24.530187+0000","last_peered":"2024-04-16T15:41:24.530187+0000","last_clean":"2024-04-16T15:41:24.530187+0000","last_became_active":"2024-04-16T15:41:23.521852+0000","last_became_peered":"2024-04-16T15:41:23.521852+0000","last_unstale":"2024-04-16T15:41:24.530187+0000","last_undegraded":"2024-04-16T15:41:24.530187+0000","last_fullsized":"2024-04-16T15:41:24.530187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069255+0000","last_change":"2024-04-16T15:41:22.525713+0000","last_active":"2024-04-16T15:41:25.069255+0000","last_peered":"2024-04-16T15:41:25.069255+0000","last_clean":"2024-04-16T15:41:25.069255+0000","last_became_active":"2024-04-16T15:41:22.525423+0000","last_became_peered":"2024-04-16T15:41:22.525423+0000","last_unstale":"2024-04-16T15:41:25.069255+0000","last_undegraded":"2024-04-16T15:41:25.069255+0000","last_fullsized":"2024-04-16T15:41:25.069255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:22.255097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068828+0000","last_change":"2024-04-16T15:41:23.536118+0000","last_active":"2024-04-16T15:41:25.068828+0000","last_peered":"2024-04-16T15:41:25.068828+0000","last_clean":"2024-04-16T15:41:25.068828+0000","last_became_active":"2024-04-16T15:41:23.535863+0000","last_became_peered":"2024-04-16T15:41:23.535863+0000","last_unstale":"2024-04-16T15:41:25.068828+0000","last_undegraded":"2024-04-16T15:41:25.068828+0000","last_fullsized":"2024-04-16T15:41:25.068828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528405+0000","last_change":"2024-04-16T15:41:22.525073+0000","last_active":"2024-04-16T15:41:24.528405+0000","last_peered":"2024-04-16T15:41:24.528405+0000","last_clean":"2024-04-16T15:41:24.528405+0000","last_became_active":"2024-04-16T15:41:22.524464+0000","last_became_peered":"2024-04-16T15:41:22.524464+0000","last_unstale":"2024-04-16T15:41:24.528405+0000","last_undegraded":"2024-04-16T15:41:24.528405+0000","last_fullsized":"2024-04-16T15:41:24.528405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529022+0000","last_change":"2024-04-16T15:41:23.526852+0000","last_active":"2024-04-16T15:41:24.529022+0000","last_peered":"2024-04-16T15:41:24.529022+0000","last_clean":"2024-04-16T15:41:24.529022+0000","last_became_active":"2024-04-16T15:41:23.526580+0000","last_became_peered":"2024-04-16T15:41:23.526580+0000","last_unstale":"2024-04-16T15:41:24.529022+0000","last_undegraded":"2024-04-16T15:41:24.529022+0000","last_fullsized":"2024-04-16T15:41:24.529022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070264+0000","last_change":"2024-04-16T15:41:22.523803+0000","last_active":"2024-04-16T15:41:25.070264+0000","last_peered":"2024-04-16T15:41:25.070264+0000","last_clean":"2024-04-16T15:41:25.070264+0000","last_became_active":"2024-04-16T15:41:22.523524+0000","last_became_peered":"2024-04-16T15:41:22.523524+0000","last_unstale":"2024-04-16T15:41:25.070264+0000","last_undegraded":"2024-04-16T15:41:25.070264+0000","last_fullsized":"2024-04-16T15:41:25.070264+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:46.457169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530098+0000","last_change":"2024-04-16T15:41:23.521456+0000","last_active":"2024-04-16T15:41:24.530098+0000","last_peered":"2024-04-16T15:41:24.530098+0000","last_clean":"2024-04-16T15:41:24.530098+0000","last_became_active":"2024-04-16T15:41:23.521114+0000","last_became_peered":"2024-04-16T15:41:23.521114+0000","last_unstale":"2024-04-16T15:41:24.530098+0000","last_undegraded":"2024-04-16T15:41:24.530098+0000","last_fullsized":"2024-04-16T15:41:24.530098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069158+0000","last_change":"2024-04-16T15:41:22.517077+0000","last_active":"2024-04-16T15:41:25.069158+0000","last_peered":"2024-04-16T15:41:25.069158+0000","last_clean":"2024-04-16T15:41:25.069158+0000","last_became_active":"2024-04-16T15:41:22.516776+0000","last_became_peered":"2024-04-16T15:41:22.516776+0000","last_unstale":"2024-04-16T15:41:25.069158+0000","last_undegraded":"2024-04-16T15:41:25.069158+0000","last_fullsized":"2024-04-16T15:41:25.069158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070000+0000","last_change":"2024-04-16T15:41:23.521606+0000","last_active":"2024-04-16T15:41:25.070000+0000","last_peered":"2024-04-16T15:41:25.070000+0000","last_clean":"2024-04-16T15:41:25.070000+0000","last_became_active":"2024-04-16T15:41:23.521351+0000","last_became_peered":"2024-04-16T15:41:23.521351+0000","last_unstale":"2024-04-16T15:41:25.070000+0000","last_undegraded":"2024-04-16T15:41:25.070000+0000","last_fullsized":"2024-04-16T15:41:25.070000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069018+0000","last_change":"2024-04-16T15:41:22.517838+0000","last_active":"2024-04-16T15:41:25.069018+0000","last_peered":"2024-04-16T15:41:25.069018+0000","last_clean":"2024-04-16T15:41:25.069018+0000","last_became_active":"2024-04-16T15:41:22.517494+0000","last_became_peered":"2024-04-16T15:41:22.517494+0000","last_unstale":"2024-04-16T15:41:25.069018+0000","last_undegraded":"2024-04-16T15:41:25.069018+0000","last_fullsized":"2024-04-16T15:41:25.069018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:50.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070060+0000","last_change":"2024-04-16T15:41:23.524122+0000","last_active":"2024-04-16T15:41:25.070060+0000","last_peered":"2024-04-16T15:41:25.070060+0000","last_clean":"2024-04-16T15:41:25.070060+0000","last_became_active":"2024-04-16T15:41:23.523821+0000","last_became_peered":"2024-04-16T15:41:23.523821+0000","last_unstale":"2024-04-16T15:41:25.070060+0000","last_undegraded":"2024-04-16T15:41:25.070060+0000","last_fullsized":"2024-04-16T15:41:25.070060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068589+0000","last_change":"2024-04-16T15:41:22.519275+0000","last_active":"2024-04-16T15:41:25.068589+0000","last_peered":"2024-04-16T15:41:25.068589+0000","last_clean":"2024-04-16T15:41:25.068589+0000","last_became_active":"2024-04-16T15:41:22.518999+0000","last_became_peered":"2024-04-16T15:41:22.518999+0000","last_unstale":"2024-04-16T15:41:25.068589+0000","last_undegraded":"2024-04-16T15:41:25.068589+0000","last_fullsized":"2024-04-16T15:41:25.068589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683431+0000","last_change":"2024-04-16T15:41:23.522582+0000","last_active":"2024-04-16T15:41:24.683431+0000","last_peered":"2024-04-16T15:41:24.683431+0000","last_clean":"2024-04-16T15:41:24.683431+0000","last_became_active":"2024-04-16T15:41:23.522254+0000","last_became_peered":"2024-04-16T15:41:23.522254+0000","last_unstale":"2024-04-16T15:41:24.683431+0000","last_undegraded":"2024-04-16T15:41:24.683431+0000","last_fullsized":"2024-04-16T15:41:24.683431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.525440+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.523261+0000","last_became_peered":"2024-04-16T15:41:22.523261+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:23.526801+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:23.526499+0000","last_became_peered":"2024-04-16T15:41:23.526499+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529584+0000","last_change":"2024-04-16T15:41:22.521239+0000","last_active":"2024-04-16T15:41:24.529584+0000","last_peered":"2024-04-16T15:41:24.529584+0000","last_clean":"2024-04-16T15:41:24.529584+0000","last_became_active":"2024-04-16T15:41:22.520944+0000","last_became_peered":"2024-04-16T15:41:22.520944+0000","last_unstale":"2024-04-16T15:41:24.529584+0000","last_undegraded":"2024-04-16T15:41:24.529584+0000","last_fullsized":"2024-04-16T15:41:24.529584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528112+0000","last_change":"2024-04-16T15:41:23.532440+0000","last_active":"2024-04-16T15:41:24.528112+0000","last_peered":"2024-04-16T15:41:24.528112+0000","last_clean":"2024-04-16T15:41:24.528112+0000","last_became_active":"2024-04-16T15:41:23.530755+0000","last_became_peered":"2024-04-16T15:41:23.530755+0000","last_unstale":"2024-04-16T15:41:24.528112+0000","last_undegraded":"2024-04-16T15:41:24.528112+0000","last_fullsized":"2024-04-16T15:41:24.528112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528231+0000","last_change":"2024-04-16T15:41:22.523523+0000","last_active":"2024-04-16T15:41:24.528231+0000","last_peered":"2024-04-16T15:41:24.528231+0000","last_clean":"2024-04-16T15:41:24.528231+0000","last_became_active":"2024-04-16T15:41:22.522766+0000","last_became_peered":"2024-04-16T15:41:22.522766+0000","last_unstale":"2024-04-16T15:41:24.528231+0000","last_undegraded":"2024-04-16T15:41:24.528231+0000","last_fullsized":"2024-04-16T15:41:24.528231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527623+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:41:24.527623+0000","last_peered":"2024-04-16T15:41:24.527623+0000","last_clean":"2024-04-16T15:41:24.527623+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:41:24.527623+0000","last_undegraded":"2024-04-16T15:41:24.527623+0000","last_fullsized":"2024-04-16T15:41:24.527623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":191844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":780600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:41:29.730 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:41:29.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.871+0000 7f237bfd6640 1 Processor -- start 2024-04-16T15:41:29.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.871+0000 7f237bfd6640 1 -- start start 2024-04-16T15:41:29.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f237bfd6640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2374153220 0x7f23740c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f237bfd6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2374159200 0x7f2374159650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f237bfd6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2374072ae0 con 0x7f23740c3270 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f237bfd6640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f23740c3780 con 0x7f2374153220 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f237bfd6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2374072dc0 con 0x7f2374159200 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f2379d72640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2374153220 0x7f23740c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f2379d72640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2374153220 0x7f23740c2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53676/0 (socket says 172.21.15.71:53676) 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f2379d72640 1 -- 172.21.15.71:0/1623680914 learned_addr learned my addr 172.21.15.71:0/1623680914 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f237a573640 1 --2- 172.21.15.71:0/1623680914 >> v2:172.21.15.122:3300/0 conn(0x7f2374159200 0x7f2374159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f2379d72640 1 -- 172.21.15.71:0/1623680914 >> v2:172.21.15.122:3300/0 conn(0x7f2374159200 msgr2=0x7f2374159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:29.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f2379d72640 1 --2- 172.21.15.71:0/1623680914 >> v2:172.21.15.122:3300/0 conn(0x7f2374159200 0x7f2374159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f2379d72640 1 -- 172.21.15.71:0/1623680914 >> v1:172.21.15.122:6789/0 conn(0x7f23740c3270 legacy=0x7f237406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.872+0000 7f2379d72640 1 -- 172.21.15.71:0/1623680914 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23740730a0 con 0x7f2374153220 2024-04-16T15:41:29.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.873+0000 7f2379d72640 1 --2- 172.21.15.71:0/1623680914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f23740c2d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f23740aa500 tx=0x7f237002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea99bc167bcd49a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:29.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.873+0000 7f2378d70640 1 -- 172.21.15.71:0/1623680914 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370035020 con 0x7f2374153220 2024-04-16T15:41:29.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.873+0000 7f2378d70640 1 -- 172.21.15.71:0/1623680914 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2370031870 con 0x7f2374153220 2024-04-16T15:41:29.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.873+0000 7f2378d70640 1 -- 172.21.15.71:0/1623680914 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370031c40 con 0x7f2374153220 2024-04-16T15:41:29.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.873+0000 7f237bfd6640 1 -- 172.21.15.71:0/1623680914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 msgr2=0x7f23740c2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.873+0000 7f237bfd6640 1 --2- 172.21.15.71:0/1623680914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f23740c2d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f23740aa500 tx=0x7f237002b910 comp rx=0 tx=0).stop 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.874+0000 7f237bfd6640 1 -- 172.21.15.71:0/1623680914 shutdown_connections 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.874+0000 7f237bfd6640 1 --2- 172.21.15.71:0/1623680914 >> v2:172.21.15.122:3300/0 conn(0x7f2374159200 0x7f2374159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.874+0000 7f237bfd6640 1 --2- 172.21.15.71:0/1623680914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f23740c2d30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.874+0000 7f237bfd6640 1 -- 172.21.15.71:0/1623680914 >> 172.21.15.71:0/1623680914 conn(0x7f23740c4200 msgr2=0x7f23740c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.874+0000 7f237bfd6640 1 -- 172.21.15.71:0/1623680914 shutdown_connections 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.874+0000 7f237bfd6640 1 -- 172.21.15.71:0/1623680914 wait complete. 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f237bfd6640 1 Processor -- start 2024-04-16T15:41:29.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f237bfd6640 1 -- start start 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f237bfd6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23740c3270 0x7f23740ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f237bfd6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f237414d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f237bfd6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2374159200 0x7f2374141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f237bfd6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f237406d050 con 0x7f2374153220 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f237bfd6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2374153600 con 0x7f2374159200 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f237bfd6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f237406d1d0 con 0x7f23740c3270 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f2379571640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f237414d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f2379571640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f237414d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53692/0 (socket says 172.21.15.71:53692) 2024-04-16T15:41:29.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.875+0000 7f2379571640 1 -- 172.21.15.71:0/2881980713 learned_addr learned my addr 172.21.15.71:0/2881980713 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f237a573640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2374159200 0x7f2374141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f2379d72640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23740c3270 0x7f23740ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f2379571640 1 -- 172.21.15.71:0/2881980713 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23740c3270 msgr2=0x7f23740ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f2379571640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23740c3270 0x7f23740ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f2379571640 1 -- 172.21.15.71:0/2881980713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2374159200 msgr2=0x7f2374141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f2379571640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2374159200 0x7f2374141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f2379571640 1 -- 172.21.15.71:0/2881980713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23741ef710 con 0x7f2374153220 2024-04-16T15:41:29.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f2379571640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f237414d730 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f236400c470 tx=0x7f236400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:29.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f236a7fc640 1 -- 172.21.15.71:0/2881980713 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23640052d0 con 0x7f2374153220 2024-04-16T15:41:29.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f236a7fc640 1 -- 172.21.15.71:0/2881980713 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2364005490 con 0x7f2374153220 2024-04-16T15:41:29.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23740aa6d0 con 0x7f2374153220 2024-04-16T15:41:29.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f236a7fc640 1 -- 172.21.15.71:0/2881980713 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364007260 con 0x7f2374153220 2024-04-16T15:41:29.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.876+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237414dc70 con 0x7f2374153220 2024-04-16T15:41:29.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.877+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2374068d20 con 0x7f2374153220 2024-04-16T15:41:29.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.878+0000 7f236a7fc640 1 -- 172.21.15.71:0/2881980713 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2364007400 con 0x7f2374153220 2024-04-16T15:41:29.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.878+0000 7f236a7fc640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f235806c8f0 0x7f235806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:29.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.878+0000 7f2379d72640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f235806c8f0 0x7f235806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:29.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.878+0000 7f236a7fc640 1 -- 172.21.15.71:0/2881980713 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f236408b0f0 con 0x7f2374153220 2024-04-16T15:41:29.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.879+0000 7f2379d72640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f235806c8f0 0x7f235806edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2370036000 tx=0x7f2370008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:29.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:29.882+0000 7f236a7fc640 1 -- 172.21.15.71:0/2881980713 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2364059420 con 0x7f2374153220 2024-04-16T15:41:30.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.057+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23741ef9f0 con 0x7f235806c8f0 2024-04-16T15:41:30.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.063+0000 7f236a7fc640 1 -- 172.21.15.71:0/2881980713 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f23741ef9f0 con 0x7f235806c8f0 2024-04-16T15:41:30.067 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:30.070 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:41:30.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f235806c8f0 msgr2=0x7f235806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f235806c8f0 0x7f235806edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2370036000 tx=0x7f2370008030 comp rx=0 tx=0).stop 2024-04-16T15:41:30.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 msgr2=0x7f237414d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f237414d730 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f236400c470 tx=0x7f236400c930 comp rx=0 tx=0).stop 2024-04-16T15:41:30.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 shutdown_connections 2024-04-16T15:41:30.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f235806c8f0 0x7f235806edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2374159200 0x7f2374141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2374153220 0x7f237414d730 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 --2- 172.21.15.71:0/2881980713 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23740c3270 0x7f23740ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 >> 172.21.15.71:0/2881980713 conn(0x7f23740c4200 msgr2=0x7f237406f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:30.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 shutdown_connections 2024-04-16T15:41:30.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.066+0000 7f237bfd6640 1 -- 172.21.15.71:0/2881980713 wait complete. 2024-04-16T15:41:30.079 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-04-16T15:41:29.713716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220816,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753904,"statfs":{"total":773094113280,"available":772867997696,"internally_reserved":0,"allocated":4759552,"data_stored":2707140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.209338"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530182+0000","last_change":"2024-04-16T15:41:23.523079+0000","last_active":"2024-04-16T15:41:24.530182+0000","last_peered":"2024-04-16T15:41:24.530182+0000","last_clean":"2024-04-16T15:41:24.530182+0000","last_became_active":"2024-04-16T15:41:23.522804+0000","last_became_peered":"2024-04-16T15:41:23.522804+0000","last_unstale":"2024-04-16T15:41:24.530182+0000","last_undegraded":"2024-04-16T15:41:24.530182+0000","last_fullsized":"2024-04-16T15:41:24.530182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530026+0000","last_change":"2024-04-16T15:41:23.518347+0000","last_active":"2024-04-16T15:41:24.530026+0000","last_peered":"2024-04-16T15:41:24.530026+0000","last_clean":"2024-04-16T15:41:24.530026+0000","last_became_active":"2024-04-16T15:41:23.517372+0000","last_became_peered":"2024-04-16T15:41:23.517372+0000","last_unstale":"2024-04-16T15:41:24.530026+0000","last_undegraded":"2024-04-16T15:41:24.530026+0000","last_fullsized":"2024-04-16T15:41:24.530026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.533697+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.533520+0000","last_became_peered":"2024-04-16T15:41:23.533520+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069345+0000","last_change":"2024-04-16T15:41:23.521655+0000","last_active":"2024-04-16T15:41:25.069345+0000","last_peered":"2024-04-16T15:41:25.069345+0000","last_clean":"2024-04-16T15:41:25.069345+0000","last_became_active":"2024-04-16T15:41:23.521419+0000","last_became_peered":"2024-04-16T15:41:23.521419+0000","last_unstale":"2024-04-16T15:41:25.069345+0000","last_undegraded":"2024-04-16T15:41:25.069345+0000","last_fullsized":"2024-04-16T15:41:25.069345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:48.607343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069423+0000","last_change":"2024-04-16T15:41:23.522246+0000","last_active":"2024-04-16T15:41:25.069423+0000","last_peered":"2024-04-16T15:41:25.069423+0000","last_clean":"2024-04-16T15:41:25.069423+0000","last_became_active":"2024-04-16T15:41:23.522009+0000","last_became_peered":"2024-04-16T15:41:23.522009+0000","last_unstale":"2024-04-16T15:41:25.069423+0000","last_undegraded":"2024-04-16T15:41:25.069423+0000","last_fullsized":"2024-04-16T15:41:25.069423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:33.728092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069091+0000","last_change":"2024-04-16T15:41:23.523004+0000","last_active":"2024-04-16T15:41:25.069091+0000","last_peered":"2024-04-16T15:41:25.069091+0000","last_clean":"2024-04-16T15:41:25.069091+0000","last_became_active":"2024-04-16T15:41:23.522742+0000","last_became_peered":"2024-04-16T15:41:23.522742+0000","last_unstale":"2024-04-16T15:41:25.069091+0000","last_undegraded":"2024-04-16T15:41:25.069091+0000","last_fullsized":"2024-04-16T15:41:25.069091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527561+0000","last_change":"2024-04-16T15:41:23.523305+0000","last_active":"2024-04-16T15:41:24.527561+0000","last_peered":"2024-04-16T15:41:24.527561+0000","last_clean":"2024-04-16T15:41:24.527561+0000","last_became_active":"2024-04-16T15:41:23.523022+0000","last_became_peered":"2024-04-16T15:41:23.523022+0000","last_unstale":"2024-04-16T15:41:24.527561+0000","last_undegraded":"2024-04-16T15:41:24.527561+0000","last_fullsized":"2024-04-16T15:41:24.527561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529666+0000","last_change":"2024-04-16T15:41:23.535446+0000","last_active":"2024-04-16T15:41:24.529666+0000","last_peered":"2024-04-16T15:41:24.529666+0000","last_clean":"2024-04-16T15:41:24.529666+0000","last_became_active":"2024-04-16T15:41:23.535156+0000","last_became_peered":"2024-04-16T15:41:23.535156+0000","last_unstale":"2024-04-16T15:41:24.529666+0000","last_undegraded":"2024-04-16T15:41:24.529666+0000","last_fullsized":"2024-04-16T15:41:24.529666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069784+0000","last_change":"2024-04-16T15:41:23.535019+0000","last_active":"2024-04-16T15:41:25.069784+0000","last_peered":"2024-04-16T15:41:25.069784+0000","last_clean":"2024-04-16T15:41:25.069784+0000","last_became_active":"2024-04-16T15:41:23.534726+0000","last_became_peered":"2024-04-16T15:41:23.534726+0000","last_unstale":"2024-04-16T15:41:25.069784+0000","last_undegraded":"2024-04-16T15:41:25.069784+0000","last_fullsized":"2024-04-16T15:41:25.069784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528288+0000","last_change":"2024-04-16T15:41:23.522709+0000","last_active":"2024-04-16T15:41:24.528288+0000","last_peered":"2024-04-16T15:41:24.528288+0000","last_clean":"2024-04-16T15:41:24.528288+0000","last_became_active":"2024-04-16T15:41:23.522437+0000","last_became_peered":"2024-04-16T15:41:23.522437+0000","last_unstale":"2024-04-16T15:41:24.528288+0000","last_undegraded":"2024-04-16T15:41:24.528288+0000","last_fullsized":"2024-04-16T15:41:24.528288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528764+0000","last_change":"2024-04-16T15:41:23.530620+0000","last_active":"2024-04-16T15:41:24.528764+0000","last_peered":"2024-04-16T15:41:24.528764+0000","last_clean":"2024-04-16T15:41:24.528764+0000","last_became_active":"2024-04-16T15:41:23.530320+0000","last_became_peered":"2024-04-16T15:41:23.530320+0000","last_unstale":"2024-04-16T15:41:24.528764+0000","last_undegraded":"2024-04-16T15:41:24.528764+0000","last_fullsized":"2024-04-16T15:41:24.528764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068927+0000","last_change":"2024-04-16T15:41:23.523419+0000","last_active":"2024-04-16T15:41:25.068927+0000","last_peered":"2024-04-16T15:41:25.068927+0000","last_clean":"2024-04-16T15:41:25.068927+0000","last_became_active":"2024-04-16T15:41:23.523081+0000","last_became_peered":"2024-04-16T15:41:23.523081+0000","last_unstale":"2024-04-16T15:41:25.068927+0000","last_undegraded":"2024-04-16T15:41:25.068927+0000","last_fullsized":"2024-04-16T15:41:25.068927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527828+0000","last_change":"2024-04-16T15:41:23.522227+0000","last_active":"2024-04-16T15:41:24.527828+0000","last_peered":"2024-04-16T15:41:24.527828+0000","last_clean":"2024-04-16T15:41:24.527828+0000","last_became_active":"2024-04-16T15:41:23.521531+0000","last_became_peered":"2024-04-16T15:41:23.521531+0000","last_unstale":"2024-04-16T15:41:24.527828+0000","last_undegraded":"2024-04-16T15:41:24.527828+0000","last_fullsized":"2024-04-16T15:41:24.527828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683427+0000","last_change":"2024-04-16T15:41:23.529710+0000","last_active":"2024-04-16T15:41:24.683427+0000","last_peered":"2024-04-16T15:41:24.683427+0000","last_clean":"2024-04-16T15:41:24.683427+0000","last_became_active":"2024-04-16T15:41:23.528557+0000","last_became_peered":"2024-04-16T15:41:23.528557+0000","last_unstale":"2024-04-16T15:41:24.683427+0000","last_undegraded":"2024-04-16T15:41:24.683427+0000","last_fullsized":"2024-04-16T15:41:24.683427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527689+0000","last_change":"2024-04-16T15:41:23.513641+0000","last_active":"2024-04-16T15:41:24.527689+0000","last_peered":"2024-04-16T15:41:24.527689+0000","last_clean":"2024-04-16T15:41:24.527689+0000","last_became_active":"2024-04-16T15:41:23.513340+0000","last_became_peered":"2024-04-16T15:41:23.513340+0000","last_unstale":"2024-04-16T15:41:24.527689+0000","last_undegraded":"2024-04-16T15:41:24.527689+0000","last_fullsized":"2024-04-16T15:41:24.527689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527345+0000","last_change":"2024-04-16T15:41:23.535782+0000","last_active":"2024-04-16T15:41:24.527345+0000","last_peered":"2024-04-16T15:41:24.527345+0000","last_clean":"2024-04-16T15:41:24.527345+0000","last_became_active":"2024-04-16T15:41:23.535549+0000","last_became_peered":"2024-04-16T15:41:23.535549+0000","last_unstale":"2024-04-16T15:41:24.527345+0000","last_undegraded":"2024-04-16T15:41:24.527345+0000","last_fullsized":"2024-04-16T15:41:24.527345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528422+0000","last_change":"2024-04-16T15:41:23.536402+0000","last_active":"2024-04-16T15:41:24.528422+0000","last_peered":"2024-04-16T15:41:24.528422+0000","last_clean":"2024-04-16T15:41:24.528422+0000","last_became_active":"2024-04-16T15:41:23.536009+0000","last_became_peered":"2024-04-16T15:41:23.536009+0000","last_unstale":"2024-04-16T15:41:24.528422+0000","last_undegraded":"2024-04-16T15:41:24.528422+0000","last_fullsized":"2024-04-16T15:41:24.528422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068843+0000","last_change":"2024-04-16T15:41:23.523701+0000","last_active":"2024-04-16T15:41:25.068843+0000","last_peered":"2024-04-16T15:41:25.068843+0000","last_clean":"2024-04-16T15:41:25.068843+0000","last_became_active":"2024-04-16T15:41:23.523442+0000","last_became_peered":"2024-04-16T15:41:23.523442+0000","last_unstale":"2024-04-16T15:41:25.068843+0000","last_undegraded":"2024-04-16T15:41:25.068843+0000","last_fullsized":"2024-04-16T15:41:25.068843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070293+0000","last_change":"2024-04-16T15:41:23.519553+0000","last_active":"2024-04-16T15:41:25.070293+0000","last_peered":"2024-04-16T15:41:25.070293+0000","last_clean":"2024-04-16T15:41:25.070293+0000","last_became_active":"2024-04-16T15:41:23.519258+0000","last_became_peered":"2024-04-16T15:41:23.519258+0000","last_unstale":"2024-04-16T15:41:25.070293+0000","last_undegraded":"2024-04-16T15:41:25.070293+0000","last_fullsized":"2024-04-16T15:41:25.070293+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:51.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068696+0000","last_change":"2024-04-16T15:41:23.522212+0000","last_active":"2024-04-16T15:41:25.068696+0000","last_peered":"2024-04-16T15:41:25.068696+0000","last_clean":"2024-04-16T15:41:25.068696+0000","last_became_active":"2024-04-16T15:41:23.521934+0000","last_became_peered":"2024-04-16T15:41:23.521934+0000","last_unstale":"2024-04-16T15:41:25.068696+0000","last_undegraded":"2024-04-16T15:41:25.068696+0000","last_fullsized":"2024-04-16T15:41:25.068696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:03.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527973+0000","last_change":"2024-04-16T15:41:23.520977+0000","last_active":"2024-04-16T15:41:24.527973+0000","last_peered":"2024-04-16T15:41:24.527973+0000","last_clean":"2024-04-16T15:41:24.527973+0000","last_became_active":"2024-04-16T15:41:23.520677+0000","last_became_peered":"2024-04-16T15:41:23.520677+0000","last_unstale":"2024-04-16T15:41:24.527973+0000","last_undegraded":"2024-04-16T15:41:24.527973+0000","last_fullsized":"2024-04-16T15:41:24.527973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683729+0000","last_change":"2024-04-16T15:41:23.527968+0000","last_active":"2024-04-16T15:41:24.683729+0000","last_peered":"2024-04-16T15:41:24.683729+0000","last_clean":"2024-04-16T15:41:24.683729+0000","last_became_active":"2024-04-16T15:41:23.527690+0000","last_became_peered":"2024-04-16T15:41:23.527690+0000","last_unstale":"2024-04-16T15:41:24.683729+0000","last_undegraded":"2024-04-16T15:41:24.683729+0000","last_fullsized":"2024-04-16T15:41:24.683729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527416+0000","last_change":"2024-04-16T15:41:23.527691+0000","last_active":"2024-04-16T15:41:24.527416+0000","last_peered":"2024-04-16T15:41:24.527416+0000","last_clean":"2024-04-16T15:41:24.527416+0000","last_became_active":"2024-04-16T15:41:23.527372+0000","last_became_peered":"2024-04-16T15:41:23.527372+0000","last_unstale":"2024-04-16T15:41:24.527416+0000","last_undegraded":"2024-04-16T15:41:24.527416+0000","last_fullsized":"2024-04-16T15:41:24.527416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069312+0000","last_change":"2024-04-16T15:41:23.525188+0000","last_active":"2024-04-16T15:41:25.069312+0000","last_peered":"2024-04-16T15:41:25.069312+0000","last_clean":"2024-04-16T15:41:25.069312+0000","last_became_active":"2024-04-16T15:41:23.524939+0000","last_became_peered":"2024-04-16T15:41:23.524939+0000","last_unstale":"2024-04-16T15:41:25.069312+0000","last_undegraded":"2024-04-16T15:41:25.069312+0000","last_fullsized":"2024-04-16T15:41:25.069312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:53.692430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068745+0000","last_change":"2024-04-16T15:41:23.526266+0000","last_active":"2024-04-16T15:41:25.068745+0000","last_peered":"2024-04-16T15:41:25.068745+0000","last_clean":"2024-04-16T15:41:25.068745+0000","last_became_active":"2024-04-16T15:41:23.525999+0000","last_became_peered":"2024-04-16T15:41:23.525999+0000","last_unstale":"2024-04-16T15:41:25.068745+0000","last_undegraded":"2024-04-16T15:41:25.068745+0000","last_fullsized":"2024-04-16T15:41:25.068745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:38.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069155+0000","last_change":"2024-04-16T15:41:23.535525+0000","last_active":"2024-04-16T15:41:25.069155+0000","last_peered":"2024-04-16T15:41:25.069155+0000","last_clean":"2024-04-16T15:41:25.069155+0000","last_became_active":"2024-04-16T15:41:23.535192+0000","last_became_peered":"2024-04-16T15:41:23.535192+0000","last_unstale":"2024-04-16T15:41:25.069155+0000","last_undegraded":"2024-04-16T15:41:25.069155+0000","last_fullsized":"2024-04-16T15:41:25.069155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068809+0000","last_change":"2024-04-16T15:41:23.527164+0000","last_active":"2024-04-16T15:41:25.068809+0000","last_peered":"2024-04-16T15:41:25.068809+0000","last_clean":"2024-04-16T15:41:25.068809+0000","last_became_active":"2024-04-16T15:41:23.526765+0000","last_became_peered":"2024-04-16T15:41:23.526765+0000","last_unstale":"2024-04-16T15:41:25.068809+0000","last_undegraded":"2024-04-16T15:41:25.068809+0000","last_fullsized":"2024-04-16T15:41:25.068809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528932+0000","last_change":"2024-04-16T15:41:23.529704+0000","last_active":"2024-04-16T15:41:24.528932+0000","last_peered":"2024-04-16T15:41:24.528932+0000","last_clean":"2024-04-16T15:41:24.528932+0000","last_became_active":"2024-04-16T15:41:23.525283+0000","last_became_peered":"2024-04-16T15:41:23.525283+0000","last_unstale":"2024-04-16T15:41:24.528932+0000","last_undegraded":"2024-04-16T15:41:24.528932+0000","last_fullsized":"2024-04-16T15:41:24.528932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068718+0000","last_change":"2024-04-16T15:41:23.523057+0000","last_active":"2024-04-16T15:41:25.068718+0000","last_peered":"2024-04-16T15:41:25.068718+0000","last_clean":"2024-04-16T15:41:25.068718+0000","last_became_active":"2024-04-16T15:41:23.522762+0000","last_became_peered":"2024-04-16T15:41:23.522762+0000","last_unstale":"2024-04-16T15:41:25.068718+0000","last_undegraded":"2024-04-16T15:41:25.068718+0000","last_fullsized":"2024-04-16T15:41:25.068718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:57.277773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068960+0000","last_change":"2024-04-16T15:41:23.522445+0000","last_active":"2024-04-16T15:41:25.068960+0000","last_peered":"2024-04-16T15:41:25.068960+0000","last_clean":"2024-04-16T15:41:25.068960+0000","last_became_active":"2024-04-16T15:41:23.522179+0000","last_became_peered":"2024-04-16T15:41:23.522179+0000","last_unstale":"2024-04-16T15:41:25.068960+0000","last_undegraded":"2024-04-16T15:41:25.068960+0000","last_fullsized":"2024-04-16T15:41:25.068960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527557+0000","last_change":"2024-04-16T15:41:23.528582+0000","last_active":"2024-04-16T15:41:24.527557+0000","last_peered":"2024-04-16T15:41:24.527557+0000","last_clean":"2024-04-16T15:41:24.527557+0000","last_became_active":"2024-04-16T15:41:23.528304+0000","last_became_peered":"2024-04-16T15:41:23.528304+0000","last_unstale":"2024-04-16T15:41:24.527557+0000","last_undegraded":"2024-04-16T15:41:24.527557+0000","last_fullsized":"2024-04-16T15:41:24.527557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:58:58.124408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069657+0000","last_change":"2024-04-16T15:41:23.519347+0000","last_active":"2024-04-16T15:41:25.069657+0000","last_peered":"2024-04-16T15:41:25.069657+0000","last_clean":"2024-04-16T15:41:25.069657+0000","last_became_active":"2024-04-16T15:41:23.519092+0000","last_became_peered":"2024-04-16T15:41:23.519092+0000","last_unstale":"2024-04-16T15:41:25.069657+0000","last_undegraded":"2024-04-16T15:41:25.069657+0000","last_fullsized":"2024-04-16T15:41:25.069657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527966+0000","last_change":"2024-04-16T15:41:23.536251+0000","last_active":"2024-04-16T15:41:24.527966+0000","last_peered":"2024-04-16T15:41:24.527966+0000","last_clean":"2024-04-16T15:41:24.527966+0000","last_became_active":"2024-04-16T15:41:23.535963+0000","last_became_peered":"2024-04-16T15:41:23.535963+0000","last_unstale":"2024-04-16T15:41:24.527966+0000","last_undegraded":"2024-04-16T15:41:24.527966+0000","last_fullsized":"2024-04-16T15:41:24.527966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068769+0000","last_change":"2024-04-16T15:41:22.522796+0000","last_active":"2024-04-16T15:41:25.068769+0000","last_peered":"2024-04-16T15:41:25.068769+0000","last_clean":"2024-04-16T15:41:25.068769+0000","last_became_active":"2024-04-16T15:41:22.522542+0000","last_became_peered":"2024-04-16T15:41:22.522542+0000","last_unstale":"2024-04-16T15:41:25.068769+0000","last_undegraded":"2024-04-16T15:41:25.068769+0000","last_fullsized":"2024-04-16T15:41:25.068769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069565+0000","last_change":"2024-04-16T15:41:23.536192+0000","last_active":"2024-04-16T15:41:25.069565+0000","last_peered":"2024-04-16T15:41:25.069565+0000","last_clean":"2024-04-16T15:41:25.069565+0000","last_became_active":"2024-04-16T15:41:23.535920+0000","last_became_peered":"2024-04-16T15:41:23.535920+0000","last_unstale":"2024-04-16T15:41:25.069565+0000","last_undegraded":"2024-04-16T15:41:25.069565+0000","last_fullsized":"2024-04-16T15:41:25.069565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069482+0000","last_change":"2024-04-16T15:41:22.523464+0000","last_active":"2024-04-16T15:41:25.069482+0000","last_peered":"2024-04-16T15:41:25.069482+0000","last_clean":"2024-04-16T15:41:25.069482+0000","last_became_active":"2024-04-16T15:41:22.523128+0000","last_became_peered":"2024-04-16T15:41:22.523128+0000","last_unstale":"2024-04-16T15:41:25.069482+0000","last_undegraded":"2024-04-16T15:41:25.069482+0000","last_fullsized":"2024-04-16T15:41:25.069482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683674+0000","last_change":"2024-04-16T15:41:23.530373+0000","last_active":"2024-04-16T15:41:24.683674+0000","last_peered":"2024-04-16T15:41:24.683674+0000","last_clean":"2024-04-16T15:41:24.683674+0000","last_became_active":"2024-04-16T15:41:23.530115+0000","last_became_peered":"2024-04-16T15:41:23.530115+0000","last_unstale":"2024-04-16T15:41:24.683674+0000","last_undegraded":"2024-04-16T15:41:24.683674+0000","last_fullsized":"2024-04-16T15:41:24.683674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069538+0000","last_change":"2024-04-16T15:41:22.521948+0000","last_active":"2024-04-16T15:41:25.069538+0000","last_peered":"2024-04-16T15:41:25.069538+0000","last_clean":"2024-04-16T15:41:25.069538+0000","last_became_active":"2024-04-16T15:41:22.521643+0000","last_became_peered":"2024-04-16T15:41:22.521643+0000","last_unstale":"2024-04-16T15:41:25.069538+0000","last_undegraded":"2024-04-16T15:41:25.069538+0000","last_fullsized":"2024-04-16T15:41:25.069538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530028+0000","last_change":"2024-04-16T15:41:23.528874+0000","last_active":"2024-04-16T15:41:24.530028+0000","last_peered":"2024-04-16T15:41:24.530028+0000","last_clean":"2024-04-16T15:41:24.530028+0000","last_became_active":"2024-04-16T15:41:23.528569+0000","last_became_peered":"2024-04-16T15:41:23.528569+0000","last_unstale":"2024-04-16T15:41:24.530028+0000","last_undegraded":"2024-04-16T15:41:24.530028+0000","last_fullsized":"2024-04-16T15:41:24.530028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683536+0000","last_change":"2024-04-16T15:41:22.525207+0000","last_active":"2024-04-16T15:41:24.683536+0000","last_peered":"2024-04-16T15:41:24.683536+0000","last_clean":"2024-04-16T15:41:24.683536+0000","last_became_active":"2024-04-16T15:41:22.524864+0000","last_became_peered":"2024-04-16T15:41:22.524864+0000","last_unstale":"2024-04-16T15:41:24.683536+0000","last_undegraded":"2024-04-16T15:41:24.683536+0000","last_fullsized":"2024-04-16T15:41:24.683536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068873+0000","last_change":"2024-04-16T15:41:23.523861+0000","last_active":"2024-04-16T15:41:25.068873+0000","last_peered":"2024-04-16T15:41:25.068873+0000","last_clean":"2024-04-16T15:41:25.068873+0000","last_became_active":"2024-04-16T15:41:23.523587+0000","last_became_peered":"2024-04-16T15:41:23.523587+0000","last_unstale":"2024-04-16T15:41:25.068873+0000","last_undegraded":"2024-04-16T15:41:25.068873+0000","last_fullsized":"2024-04-16T15:41:25.068873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:22.526265+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:22.525849+0000","last_became_peered":"2024-04-16T15:41:22.525849+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:02.400517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528806+0000","last_change":"2024-04-16T15:41:23.521523+0000","last_active":"2024-04-16T15:41:24.528806+0000","last_peered":"2024-04-16T15:41:24.528806+0000","last_clean":"2024-04-16T15:41:24.528806+0000","last_became_active":"2024-04-16T15:41:23.520768+0000","last_became_peered":"2024-04-16T15:41:23.520768+0000","last_unstale":"2024-04-16T15:41:24.528806+0000","last_undegraded":"2024-04-16T15:41:24.528806+0000","last_fullsized":"2024-04-16T15:41:24.528806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527590+0000","last_change":"2024-04-16T15:41:22.528402+0000","last_active":"2024-04-16T15:41:24.527590+0000","last_peered":"2024-04-16T15:41:24.527590+0000","last_clean":"2024-04-16T15:41:24.527590+0000","last_became_active":"2024-04-16T15:41:22.527999+0000","last_became_peered":"2024-04-16T15:41:22.527999+0000","last_unstale":"2024-04-16T15:41:24.527590+0000","last_undegraded":"2024-04-16T15:41:24.527590+0000","last_fullsized":"2024-04-16T15:41:24.527590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683644+0000","last_change":"2024-04-16T15:41:23.535673+0000","last_active":"2024-04-16T15:41:24.683644+0000","last_peered":"2024-04-16T15:41:24.683644+0000","last_clean":"2024-04-16T15:41:24.683644+0000","last_became_active":"2024-04-16T15:41:23.535387+0000","last_became_peered":"2024-04-16T15:41:23.535387+0000","last_unstale":"2024-04-16T15:41:24.683644+0000","last_undegraded":"2024-04-16T15:41:24.683644+0000","last_fullsized":"2024-04-16T15:41:24.683644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070111+0000","last_change":"2024-04-16T15:41:22.526184+0000","last_active":"2024-04-16T15:41:25.070111+0000","last_peered":"2024-04-16T15:41:25.070111+0000","last_clean":"2024-04-16T15:41:25.070111+0000","last_became_active":"2024-04-16T15:41:22.525819+0000","last_became_peered":"2024-04-16T15:41:22.525819+0000","last_unstale":"2024-04-16T15:41:25.070111+0000","last_undegraded":"2024-04-16T15:41:25.070111+0000","last_fullsized":"2024-04-16T15:41:25.070111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530001+0000","last_change":"2024-04-16T15:41:23.522193+0000","last_active":"2024-04-16T15:41:24.530001+0000","last_peered":"2024-04-16T15:41:24.530001+0000","last_clean":"2024-04-16T15:41:24.530001+0000","last_became_active":"2024-04-16T15:41:23.521900+0000","last_became_peered":"2024-04-16T15:41:23.521900+0000","last_unstale":"2024-04-16T15:41:24.530001+0000","last_undegraded":"2024-04-16T15:41:24.530001+0000","last_fullsized":"2024-04-16T15:41:24.530001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.521519+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.521234+0000","last_became_peered":"2024-04-16T15:41:22.521234+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.521070+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.520819+0000","last_became_peered":"2024-04-16T15:41:23.520819+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:03:41.341706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530092+0000","last_change":"2024-04-16T15:41:22.519163+0000","last_active":"2024-04-16T15:41:24.530092+0000","last_peered":"2024-04-16T15:41:24.530092+0000","last_clean":"2024-04-16T15:41:24.530092+0000","last_became_active":"2024-04-16T15:41:22.518905+0000","last_became_peered":"2024-04-16T15:41:22.518905+0000","last_unstale":"2024-04-16T15:41:24.530092+0000","last_undegraded":"2024-04-16T15:41:24.530092+0000","last_fullsized":"2024-04-16T15:41:24.530092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:03.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528974+0000","last_change":"2024-04-16T15:41:23.528995+0000","last_active":"2024-04-16T15:41:24.528974+0000","last_peered":"2024-04-16T15:41:24.528974+0000","last_clean":"2024-04-16T15:41:24.528974+0000","last_became_active":"2024-04-16T15:41:23.526288+0000","last_became_peered":"2024-04-16T15:41:23.526288+0000","last_unstale":"2024-04-16T15:41:24.528974+0000","last_undegraded":"2024-04-16T15:41:24.528974+0000","last_fullsized":"2024-04-16T15:41:24.528974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528459+0000","last_change":"2024-04-16T15:41:22.525433+0000","last_active":"2024-04-16T15:41:24.528459+0000","last_peered":"2024-04-16T15:41:24.528459+0000","last_clean":"2024-04-16T15:41:24.528459+0000","last_became_active":"2024-04-16T15:41:22.524567+0000","last_became_peered":"2024-04-16T15:41:22.524567+0000","last_unstale":"2024-04-16T15:41:24.528459+0000","last_undegraded":"2024-04-16T15:41:24.528459+0000","last_fullsized":"2024-04-16T15:41:24.528459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:56.272300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069430+0000","last_change":"2024-04-16T15:41:23.522367+0000","last_active":"2024-04-16T15:41:25.069430+0000","last_peered":"2024-04-16T15:41:25.069430+0000","last_clean":"2024-04-16T15:41:25.069430+0000","last_became_active":"2024-04-16T15:41:23.521984+0000","last_became_peered":"2024-04-16T15:41:23.521984+0000","last_unstale":"2024-04-16T15:41:25.069430+0000","last_undegraded":"2024-04-16T15:41:25.069430+0000","last_fullsized":"2024-04-16T15:41:25.069430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528812+0000","last_change":"2024-04-16T15:41:22.523086+0000","last_active":"2024-04-16T15:41:24.528812+0000","last_peered":"2024-04-16T15:41:24.528812+0000","last_clean":"2024-04-16T15:41:24.528812+0000","last_became_active":"2024-04-16T15:41:22.522759+0000","last_became_peered":"2024-04-16T15:41:22.522759+0000","last_unstale":"2024-04-16T15:41:24.528812+0000","last_undegraded":"2024-04-16T15:41:24.528812+0000","last_fullsized":"2024-04-16T15:41:24.528812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683778+0000","last_change":"2024-04-16T15:41:23.529819+0000","last_active":"2024-04-16T15:41:24.683778+0000","last_peered":"2024-04-16T15:41:24.683778+0000","last_clean":"2024-04-16T15:41:24.683778+0000","last_became_active":"2024-04-16T15:41:23.529543+0000","last_became_peered":"2024-04-16T15:41:23.529543+0000","last_unstale":"2024-04-16T15:41:24.683778+0000","last_undegraded":"2024-04-16T15:41:24.683778+0000","last_fullsized":"2024-04-16T15:41:24.683778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:22.522411+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:22.522130+0000","last_became_peered":"2024-04-16T15:41:22.522130+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:41.323112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069849+0000","last_change":"2024-04-16T15:41:23.535122+0000","last_active":"2024-04-16T15:41:25.069849+0000","last_peered":"2024-04-16T15:41:25.069849+0000","last_clean":"2024-04-16T15:41:25.069849+0000","last_became_active":"2024-04-16T15:41:23.534825+0000","last_became_peered":"2024-04-16T15:41:23.534825+0000","last_unstale":"2024-04-16T15:41:25.069849+0000","last_undegraded":"2024-04-16T15:41:25.069849+0000","last_fullsized":"2024-04-16T15:41:25.069849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530105+0000","last_change":"2024-04-16T15:41:22.520252+0000","last_active":"2024-04-16T15:41:24.530105+0000","last_peered":"2024-04-16T15:41:24.530105+0000","last_clean":"2024-04-16T15:41:24.530105+0000","last_became_active":"2024-04-16T15:41:22.519829+0000","last_became_peered":"2024-04-16T15:41:22.519829+0000","last_unstale":"2024-04-16T15:41:24.530105+0000","last_undegraded":"2024-04-16T15:41:24.530105+0000","last_fullsized":"2024-04-16T15:41:24.530105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529732+0000","last_change":"2024-04-16T15:41:23.522432+0000","last_active":"2024-04-16T15:41:24.529732+0000","last_peered":"2024-04-16T15:41:24.529732+0000","last_clean":"2024-04-16T15:41:24.529732+0000","last_became_active":"2024-04-16T15:41:23.522159+0000","last_became_peered":"2024-04-16T15:41:23.522159+0000","last_unstale":"2024-04-16T15:41:24.529732+0000","last_undegraded":"2024-04-16T15:41:24.529732+0000","last_fullsized":"2024-04-16T15:41:24.529732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069145+0000","last_change":"2024-04-16T15:41:22.527064+0000","last_active":"2024-04-16T15:41:25.069145+0000","last_peered":"2024-04-16T15:41:25.069145+0000","last_clean":"2024-04-16T15:41:25.069145+0000","last_became_active":"2024-04-16T15:41:22.525811+0000","last_became_peered":"2024-04-16T15:41:22.525811+0000","last_unstale":"2024-04-16T15:41:25.069145+0000","last_undegraded":"2024-04-16T15:41:25.069145+0000","last_fullsized":"2024-04-16T15:41:25.069145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527532+0000","last_change":"2024-04-16T15:41:23.527859+0000","last_active":"2024-04-16T15:41:24.527532+0000","last_peered":"2024-04-16T15:41:24.527532+0000","last_clean":"2024-04-16T15:41:24.527532+0000","last_became_active":"2024-04-16T15:41:23.527295+0000","last_became_peered":"2024-04-16T15:41:23.527295+0000","last_unstale":"2024-04-16T15:41:24.527532+0000","last_undegraded":"2024-04-16T15:41:24.527532+0000","last_fullsized":"2024-04-16T15:41:24.527532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:00:18.934742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068514+0000","last_change":"2024-04-16T15:41:22.524475+0000","last_active":"2024-04-16T15:41:25.068514+0000","last_peered":"2024-04-16T15:41:25.068514+0000","last_clean":"2024-04-16T15:41:25.068514+0000","last_became_active":"2024-04-16T15:41:22.523303+0000","last_became_peered":"2024-04-16T15:41:22.523303+0000","last_unstale":"2024-04-16T15:41:25.068514+0000","last_undegraded":"2024-04-16T15:41:25.068514+0000","last_fullsized":"2024-04-16T15:41:25.068514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:23.523183+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:23.522845+0000","last_became_peered":"2024-04-16T15:41:23.522845+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527394+0000","last_change":"2024-04-16T15:41:22.522941+0000","last_active":"2024-04-16T15:41:24.527394+0000","last_peered":"2024-04-16T15:41:24.527394+0000","last_clean":"2024-04-16T15:41:24.527394+0000","last_became_active":"2024-04-16T15:41:22.522633+0000","last_became_peered":"2024-04-16T15:41:22.522633+0000","last_unstale":"2024-04-16T15:41:24.527394+0000","last_undegraded":"2024-04-16T15:41:24.527394+0000","last_fullsized":"2024-04-16T15:41:24.527394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529991+0000","last_change":"2024-04-16T15:41:23.528140+0000","last_active":"2024-04-16T15:41:24.529991+0000","last_peered":"2024-04-16T15:41:24.529991+0000","last_clean":"2024-04-16T15:41:24.529991+0000","last_became_active":"2024-04-16T15:41:23.527841+0000","last_became_peered":"2024-04-16T15:41:23.527841+0000","last_unstale":"2024-04-16T15:41:24.529991+0000","last_undegraded":"2024-04-16T15:41:24.529991+0000","last_fullsized":"2024-04-16T15:41:24.529991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528233+0000","last_change":"2024-04-16T15:41:22.526961+0000","last_active":"2024-04-16T15:41:24.528233+0000","last_peered":"2024-04-16T15:41:24.528233+0000","last_clean":"2024-04-16T15:41:24.528233+0000","last_became_active":"2024-04-16T15:41:22.526140+0000","last_became_peered":"2024-04-16T15:41:22.526140+0000","last_unstale":"2024-04-16T15:41:24.528233+0000","last_undegraded":"2024-04-16T15:41:24.528233+0000","last_fullsized":"2024-04-16T15:41:24.528233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528839+0000","last_change":"2024-04-16T15:41:23.521298+0000","last_active":"2024-04-16T15:41:24.528839+0000","last_peered":"2024-04-16T15:41:24.528839+0000","last_clean":"2024-04-16T15:41:24.528839+0000","last_became_active":"2024-04-16T15:41:23.521041+0000","last_became_peered":"2024-04-16T15:41:23.521041+0000","last_unstale":"2024-04-16T15:41:24.528839+0000","last_undegraded":"2024-04-16T15:41:24.528839+0000","last_fullsized":"2024-04-16T15:41:24.528839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683715+0000","last_change":"2024-04-16T15:41:22.529046+0000","last_active":"2024-04-16T15:41:24.683715+0000","last_peered":"2024-04-16T15:41:24.683715+0000","last_clean":"2024-04-16T15:41:24.683715+0000","last_became_active":"2024-04-16T15:41:22.528745+0000","last_became_peered":"2024-04-16T15:41:22.528745+0000","last_unstale":"2024-04-16T15:41:24.683715+0000","last_undegraded":"2024-04-16T15:41:24.683715+0000","last_fullsized":"2024-04-16T15:41:24.683715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683671+0000","last_change":"2024-04-16T15:41:23.522758+0000","last_active":"2024-04-16T15:41:24.683671+0000","last_peered":"2024-04-16T15:41:24.683671+0000","last_clean":"2024-04-16T15:41:24.683671+0000","last_became_active":"2024-04-16T15:41:23.521964+0000","last_became_peered":"2024-04-16T15:41:23.521964+0000","last_unstale":"2024-04-16T15:41:24.683671+0000","last_undegraded":"2024-04-16T15:41:24.683671+0000","last_fullsized":"2024-04-16T15:41:24.683671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070256+0000","last_change":"2024-04-16T15:41:22.523228+0000","last_active":"2024-04-16T15:41:25.070256+0000","last_peered":"2024-04-16T15:41:25.070256+0000","last_clean":"2024-04-16T15:41:25.070256+0000","last_became_active":"2024-04-16T15:41:22.522888+0000","last_became_peered":"2024-04-16T15:41:22.522888+0000","last_unstale":"2024-04-16T15:41:25.070256+0000","last_undegraded":"2024-04-16T15:41:25.070256+0000","last_fullsized":"2024-04-16T15:41:25.070256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:59.195492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069536+0000","last_change":"2024-04-16T15:41:23.525797+0000","last_active":"2024-04-16T15:41:25.069536+0000","last_peered":"2024-04-16T15:41:25.069536+0000","last_clean":"2024-04-16T15:41:25.069536+0000","last_became_active":"2024-04-16T15:41:23.525537+0000","last_became_peered":"2024-04-16T15:41:23.525537+0000","last_unstale":"2024-04-16T15:41:25.069536+0000","last_undegraded":"2024-04-16T15:41:25.069536+0000","last_fullsized":"2024-04-16T15:41:25.069536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:06.161198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068884+0000","last_change":"2024-04-16T15:41:22.521885+0000","last_active":"2024-04-16T15:41:25.068884+0000","last_peered":"2024-04-16T15:41:25.068884+0000","last_clean":"2024-04-16T15:41:25.068884+0000","last_became_active":"2024-04-16T15:41:22.521574+0000","last_became_peered":"2024-04-16T15:41:22.521574+0000","last_unstale":"2024-04-16T15:41:25.068884+0000","last_undegraded":"2024-04-16T15:41:25.068884+0000","last_fullsized":"2024-04-16T15:41:25.068884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527576+0000","last_change":"2024-04-16T15:41:23.529208+0000","last_active":"2024-04-16T15:41:24.527576+0000","last_peered":"2024-04-16T15:41:24.527576+0000","last_clean":"2024-04-16T15:41:24.527576+0000","last_became_active":"2024-04-16T15:41:23.528946+0000","last_became_peered":"2024-04-16T15:41:23.528946+0000","last_unstale":"2024-04-16T15:41:24.527576+0000","last_undegraded":"2024-04-16T15:41:24.527576+0000","last_fullsized":"2024-04-16T15:41:24.527576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:59:03.633363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069471+0000","last_change":"2024-04-16T15:41:22.525515+0000","last_active":"2024-04-16T15:41:25.069471+0000","last_peered":"2024-04-16T15:41:25.069471+0000","last_clean":"2024-04-16T15:41:25.069471+0000","last_became_active":"2024-04-16T15:41:22.525228+0000","last_became_peered":"2024-04-16T15:41:22.525228+0000","last_unstale":"2024-04-16T15:41:25.069471+0000","last_undegraded":"2024-04-16T15:41:25.069471+0000","last_fullsized":"2024-04-16T15:41:25.069471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:55:38.421964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068802+0000","last_change":"2024-04-16T15:41:23.522196+0000","last_active":"2024-04-16T15:41:25.068802+0000","last_peered":"2024-04-16T15:41:25.068802+0000","last_clean":"2024-04-16T15:41:25.068802+0000","last_became_active":"2024-04-16T15:41:23.521931+0000","last_became_peered":"2024-04-16T15:41:23.521931+0000","last_unstale":"2024-04-16T15:41:25.068802+0000","last_undegraded":"2024-04-16T15:41:25.068802+0000","last_fullsized":"2024-04-16T15:41:25.068802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069160+0000","last_change":"2024-04-16T15:41:22.523743+0000","last_active":"2024-04-16T15:41:25.069160+0000","last_peered":"2024-04-16T15:41:25.069160+0000","last_clean":"2024-04-16T15:41:25.069160+0000","last_became_active":"2024-04-16T15:41:22.523348+0000","last_became_peered":"2024-04-16T15:41:22.523348+0000","last_unstale":"2024-04-16T15:41:25.069160+0000","last_undegraded":"2024-04-16T15:41:25.069160+0000","last_fullsized":"2024-04-16T15:41:25.069160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:48.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528766+0000","last_change":"2024-04-16T15:41:23.524838+0000","last_active":"2024-04-16T15:41:24.528766+0000","last_peered":"2024-04-16T15:41:24.528766+0000","last_clean":"2024-04-16T15:41:24.528766+0000","last_became_active":"2024-04-16T15:41:23.524540+0000","last_became_peered":"2024-04-16T15:41:23.524540+0000","last_unstale":"2024-04-16T15:41:24.528766+0000","last_undegraded":"2024-04-16T15:41:24.528766+0000","last_fullsized":"2024-04-16T15:41:24.528766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527606+0000","last_change":"2024-04-16T15:41:22.522239+0000","last_active":"2024-04-16T15:41:24.527606+0000","last_peered":"2024-04-16T15:41:24.527606+0000","last_clean":"2024-04-16T15:41:24.527606+0000","last_became_active":"2024-04-16T15:41:22.521860+0000","last_became_peered":"2024-04-16T15:41:22.521860+0000","last_unstale":"2024-04-16T15:41:24.527606+0000","last_undegraded":"2024-04-16T15:41:24.527606+0000","last_fullsized":"2024-04-16T15:41:24.527606+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530187+0000","last_change":"2024-04-16T15:41:23.522134+0000","last_active":"2024-04-16T15:41:24.530187+0000","last_peered":"2024-04-16T15:41:24.530187+0000","last_clean":"2024-04-16T15:41:24.530187+0000","last_became_active":"2024-04-16T15:41:23.521852+0000","last_became_peered":"2024-04-16T15:41:23.521852+0000","last_unstale":"2024-04-16T15:41:24.530187+0000","last_undegraded":"2024-04-16T15:41:24.530187+0000","last_fullsized":"2024-04-16T15:41:24.530187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069255+0000","last_change":"2024-04-16T15:41:22.525713+0000","last_active":"2024-04-16T15:41:25.069255+0000","last_peered":"2024-04-16T15:41:25.069255+0000","last_clean":"2024-04-16T15:41:25.069255+0000","last_became_active":"2024-04-16T15:41:22.525423+0000","last_became_peered":"2024-04-16T15:41:22.525423+0000","last_unstale":"2024-04-16T15:41:25.069255+0000","last_undegraded":"2024-04-16T15:41:25.069255+0000","last_fullsized":"2024-04-16T15:41:25.069255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:22.255097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068828+0000","last_change":"2024-04-16T15:41:23.536118+0000","last_active":"2024-04-16T15:41:25.068828+0000","last_peered":"2024-04-16T15:41:25.068828+0000","last_clean":"2024-04-16T15:41:25.068828+0000","last_became_active":"2024-04-16T15:41:23.535863+0000","last_became_peered":"2024-04-16T15:41:23.535863+0000","last_unstale":"2024-04-16T15:41:25.068828+0000","last_undegraded":"2024-04-16T15:41:25.068828+0000","last_fullsized":"2024-04-16T15:41:25.068828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528405+0000","last_change":"2024-04-16T15:41:22.525073+0000","last_active":"2024-04-16T15:41:24.528405+0000","last_peered":"2024-04-16T15:41:24.528405+0000","last_clean":"2024-04-16T15:41:24.528405+0000","last_became_active":"2024-04-16T15:41:22.524464+0000","last_became_peered":"2024-04-16T15:41:22.524464+0000","last_unstale":"2024-04-16T15:41:24.528405+0000","last_undegraded":"2024-04-16T15:41:24.528405+0000","last_fullsized":"2024-04-16T15:41:24.528405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529022+0000","last_change":"2024-04-16T15:41:23.526852+0000","last_active":"2024-04-16T15:41:24.529022+0000","last_peered":"2024-04-16T15:41:24.529022+0000","last_clean":"2024-04-16T15:41:24.529022+0000","last_became_active":"2024-04-16T15:41:23.526580+0000","last_became_peered":"2024-04-16T15:41:23.526580+0000","last_unstale":"2024-04-16T15:41:24.529022+0000","last_undegraded":"2024-04-16T15:41:24.529022+0000","last_fullsized":"2024-04-16T15:41:24.529022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070264+0000","last_change":"2024-04-16T15:41:22.523803+0000","last_active":"2024-04-16T15:41:25.070264+0000","last_peered":"2024-04-16T15:41:25.070264+0000","last_clean":"2024-04-16T15:41:25.070264+0000","last_became_active":"2024-04-16T15:41:22.523524+0000","last_became_peered":"2024-04-16T15:41:22.523524+0000","last_unstale":"2024-04-16T15:41:25.070264+0000","last_undegraded":"2024-04-16T15:41:25.070264+0000","last_fullsized":"2024-04-16T15:41:25.070264+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:46.457169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530098+0000","last_change":"2024-04-16T15:41:23.521456+0000","last_active":"2024-04-16T15:41:24.530098+0000","last_peered":"2024-04-16T15:41:24.530098+0000","last_clean":"2024-04-16T15:41:24.530098+0000","last_became_active":"2024-04-16T15:41:23.521114+0000","last_became_peered":"2024-04-16T15:41:23.521114+0000","last_unstale":"2024-04-16T15:41:24.530098+0000","last_undegraded":"2024-04-16T15:41:24.530098+0000","last_fullsized":"2024-04-16T15:41:24.530098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069158+0000","last_change":"2024-04-16T15:41:22.517077+0000","last_active":"2024-04-16T15:41:25.069158+0000","last_peered":"2024-04-16T15:41:25.069158+0000","last_clean":"2024-04-16T15:41:25.069158+0000","last_became_active":"2024-04-16T15:41:22.516776+0000","last_became_peered":"2024-04-16T15:41:22.516776+0000","last_unstale":"2024-04-16T15:41:25.069158+0000","last_undegraded":"2024-04-16T15:41:25.069158+0000","last_fullsized":"2024-04-16T15:41:25.069158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070000+0000","last_change":"2024-04-16T15:41:23.521606+0000","last_active":"2024-04-16T15:41:25.070000+0000","last_peered":"2024-04-16T15:41:25.070000+0000","last_clean":"2024-04-16T15:41:25.070000+0000","last_became_active":"2024-04-16T15:41:23.521351+0000","last_became_peered":"2024-04-16T15:41:23.521351+0000","last_unstale":"2024-04-16T15:41:25.070000+0000","last_undegraded":"2024-04-16T15:41:25.070000+0000","last_fullsized":"2024-04-16T15:41:25.070000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069018+0000","last_change":"2024-04-16T15:41:22.517838+0000","last_active":"2024-04-16T15:41:25.069018+0000","last_peered":"2024-04-16T15:41:25.069018+0000","last_clean":"2024-04-16T15:41:25.069018+0000","last_became_active":"2024-04-16T15:41:22.517494+0000","last_became_peered":"2024-04-16T15:41:22.517494+0000","last_unstale":"2024-04-16T15:41:25.069018+0000","last_undegraded":"2024-04-16T15:41:25.069018+0000","last_fullsized":"2024-04-16T15:41:25.069018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:50.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070060+0000","last_change":"2024-04-16T15:41:23.524122+0000","last_active":"2024-04-16T15:41:25.070060+0000","last_peered":"2024-04-16T15:41:25.070060+0000","last_clean":"2024-04-16T15:41:25.070060+0000","last_became_active":"2024-04-16T15:41:23.523821+0000","last_became_peered":"2024-04-16T15:41:23.523821+0000","last_unstale":"2024-04-16T15:41:25.070060+0000","last_undegraded":"2024-04-16T15:41:25.070060+0000","last_fullsized":"2024-04-16T15:41:25.070060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068589+0000","last_change":"2024-04-16T15:41:22.519275+0000","last_active":"2024-04-16T15:41:25.068589+0000","last_peered":"2024-04-16T15:41:25.068589+0000","last_clean":"2024-04-16T15:41:25.068589+0000","last_became_active":"2024-04-16T15:41:22.518999+0000","last_became_peered":"2024-04-16T15:41:22.518999+0000","last_unstale":"2024-04-16T15:41:25.068589+0000","last_undegraded":"2024-04-16T15:41:25.068589+0000","last_fullsized":"2024-04-16T15:41:25.068589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683431+0000","last_change":"2024-04-16T15:41:23.522582+0000","last_active":"2024-04-16T15:41:24.683431+0000","last_peered":"2024-04-16T15:41:24.683431+0000","last_clean":"2024-04-16T15:41:24.683431+0000","last_became_active":"2024-04-16T15:41:23.522254+0000","last_became_peered":"2024-04-16T15:41:23.522254+0000","last_unstale":"2024-04-16T15:41:24.683431+0000","last_undegraded":"2024-04-16T15:41:24.683431+0000","last_fullsized":"2024-04-16T15:41:24.683431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.525440+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.523261+0000","last_became_peered":"2024-04-16T15:41:22.523261+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:23.526801+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:23.526499+0000","last_became_peered":"2024-04-16T15:41:23.526499+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529584+0000","last_change":"2024-04-16T15:41:22.521239+0000","last_active":"2024-04-16T15:41:24.529584+0000","last_peered":"2024-04-16T15:41:24.529584+0000","last_clean":"2024-04-16T15:41:24.529584+0000","last_became_active":"2024-04-16T15:41:22.520944+0000","last_became_peered":"2024-04-16T15:41:22.520944+0000","last_unstale":"2024-04-16T15:41:24.529584+0000","last_undegraded":"2024-04-16T15:41:24.529584+0000","last_fullsized":"2024-04-16T15:41:24.529584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528112+0000","last_change":"2024-04-16T15:41:23.532440+0000","last_active":"2024-04-16T15:41:24.528112+0000","last_peered":"2024-04-16T15:41:24.528112+0000","last_clean":"2024-04-16T15:41:24.528112+0000","last_became_active":"2024-04-16T15:41:23.530755+0000","last_became_peered":"2024-04-16T15:41:23.530755+0000","last_unstale":"2024-04-16T15:41:24.528112+0000","last_undegraded":"2024-04-16T15:41:24.528112+0000","last_fullsized":"2024-04-16T15:41:24.528112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528231+0000","last_change":"2024-04-16T15:41:22.523523+0000","last_active":"2024-04-16T15:41:24.528231+0000","last_peered":"2024-04-16T15:41:24.528231+0000","last_clean":"2024-04-16T15:41:24.528231+0000","last_became_active":"2024-04-16T15:41:22.522766+0000","last_became_peered":"2024-04-16T15:41:22.522766+0000","last_unstale":"2024-04-16T15:41:24.528231+0000","last_undegraded":"2024-04-16T15:41:24.528231+0000","last_fullsized":"2024-04-16T15:41:24.528231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527623+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:41:24.527623+0000","last_peered":"2024-04-16T15:41:24.527623+0000","last_clean":"2024-04-16T15:41:24.527623+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:41:24.527623+0000","last_undegraded":"2024-04-16T15:41:24.527623+0000","last_fullsized":"2024-04-16T15:41:24.527623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":191844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":780600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:41:30.086 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-16T15:41:30.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:41:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.224+0000 7f55db255640 1 Processor -- start 2024-04-16T15:41:30.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.224+0000 7f55db255640 1 -- start start 2024-04-16T15:41:30.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.224+0000 7f55db255640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f55d4150f40 0x7f55d40accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.224+0000 7f55db255640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d4153ba0 con 0x7f55d40ad210 2024-04-16T15:41:30.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.224+0000 7f55db255640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d4153900 con 0x7f55d40ab950 2024-04-16T15:41:30.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.224+0000 7f55db255640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f55d40ac4d0 con 0x7f55d4150f40 2024-04-16T15:41:30.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d3fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f55d40ad210 0x7f55d40aa3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35044/0 (socket says 172.21.15.71:35044) 2024-04-16T15:41:30.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d3fff640 1 -- 172.21.15.71:0/87312092 learned_addr learned my addr 172.21.15.71:0/87312092 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:30.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d8ff1640 1 --2- 172.21.15.71:0/87312092 >> v2:172.21.15.122:3300/0 conn(0x7f55d4150f40 0x7f55d40accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d37fe640 1 -- 172.21.15.71:0/87312092 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847472360 0 0) 0x7f55d4153ba0 con 0x7f55d40ad210 2024-04-16T15:41:30.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d37fe640 1 -- 172.21.15.71:0/87312092 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55c0003520 con 0x7f55d40ad210 2024-04-16T15:41:30.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d37fe640 1 -- 172.21.15.71:0/87312092 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3197437716 0 0) 0x7f55c0003520 con 0x7f55d40ad210 2024-04-16T15:41:30.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d8ff1640 1 -- 172.21.15.71:0/87312092 >> v1:172.21.15.122:6789/0 conn(0x7f55d40ab950 legacy=0x7f55d415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d8ff1640 1 -- 172.21.15.71:0/87312092 >> v1:172.21.15.71:6789/0 conn(0x7f55d40ad210 legacy=0x7f55d40aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.225+0000 7f55d8ff1640 1 -- 172.21.15.71:0/87312092 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d40ac7b0 con 0x7f55d4150f40 2024-04-16T15:41:30.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.226+0000 7f55d8ff1640 1 --2- 172.21.15.71:0/87312092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d4150f40 0x7f55d40accd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0097a0 tx=0x7f55cc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd6153a5abf3417f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:30.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.226+0000 7f55d37fe640 1 -- 172.21.15.71:0/87312092 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0019f0 con 0x7f55d4150f40 2024-04-16T15:41:30.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.226+0000 7f55d37fe640 1 -- 172.21.15.71:0/87312092 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55cc037070 con 0x7f55d4150f40 2024-04-16T15:41:30.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.226+0000 7f55d37fe640 1 -- 172.21.15.71:0/87312092 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc031790 con 0x7f55d4150f40 2024-04-16T15:41:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.227+0000 7f55db255640 1 -- 172.21.15.71:0/87312092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d4150f40 msgr2=0x7f55d40accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.227+0000 7f55db255640 1 --2- 172.21.15.71:0/87312092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d4150f40 0x7f55d40accd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0097a0 tx=0x7f55cc02b8b0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.227+0000 7f55db255640 1 -- 172.21.15.71:0/87312092 shutdown_connections 2024-04-16T15:41:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.227+0000 7f55db255640 1 --2- 172.21.15.71:0/87312092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d4150f40 0x7f55d40accd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.227+0000 7f55db255640 1 -- 172.21.15.71:0/87312092 >> 172.21.15.71:0/87312092 conn(0x7f55d40c3e10 msgr2=0x7f55d40c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.227+0000 7f55db255640 1 -- 172.21.15.71:0/87312092 shutdown_connections 2024-04-16T15:41:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.227+0000 7f55db255640 1 -- 172.21.15.71:0/87312092 wait complete. 2024-04-16T15:41:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.228+0000 7f55db255640 1 Processor -- start 2024-04-16T15:41:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.228+0000 7f55db255640 1 -- start start 2024-04-16T15:41:30.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55db255640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d40ab950 0x7f55d414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55db255640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d4150f40 0x7f55d41ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55db255640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d414a230 0x7f55d4149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55db255640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55d40aac50 con 0x7f55d40ab950 2024-04-16T15:41:30.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55db255640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55d40aaad0 con 0x7f55d414a230 2024-04-16T15:41:30.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55db255640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f55d406fdd0 con 0x7f55d4150f40 2024-04-16T15:41:30.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d40ab950 0x7f55d414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d40ab950 0x7f55d414b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53702/0 (socket says 172.21.15.71:53702) 2024-04-16T15:41:30.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 -- 172.21.15.71:0/1351101738 learned_addr learned my addr 172.21.15.71:0/1351101738 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:30.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d3fff640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d4150f40 0x7f55d41ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d97f2640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d414a230 0x7f55d4149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 -- 172.21.15.71:0/1351101738 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d4150f40 msgr2=0x7f55d41ed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d4150f40 0x7f55d41ed0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 -- 172.21.15.71:0/1351101738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d414a230 msgr2=0x7f55d4149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d414a230 0x7f55d4149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 -- 172.21.15.71:0/1351101738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d41dd3c0 con 0x7f55d40ab950 2024-04-16T15:41:30.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d8ff1640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d40ab950 0x7f55d414b610 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f55cc000c00 tx=0x7f55cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:30.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d17fa640 1 -- 172.21.15.71:0/1351101738 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc032620 con 0x7f55d40ab950 2024-04-16T15:41:30.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55d17fa640 1 -- 172.21.15.71:0/1351101738 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55cc0327c0 con 0x7f55d40ab950 2024-04-16T15:41:30.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.229+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d4155ee0 con 0x7f55d40ab950 2024-04-16T15:41:30.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.230+0000 7f55d17fa640 1 -- 172.21.15.71:0/1351101738 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc032bb0 con 0x7f55d40ab950 2024-04-16T15:41:30.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.230+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d4153400 con 0x7f55d40ab950 2024-04-16T15:41:30.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.231+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d4068d20 con 0x7f55d40ab950 2024-04-16T15:41:30.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.231+0000 7f55d17fa640 1 -- 172.21.15.71:0/1351101738 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55cc0453f0 con 0x7f55d40ab950 2024-04-16T15:41:30.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.231+0000 7f55d17fa640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f55b406c920 0x7f55b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.231+0000 7f55d17fa640 1 -- 172.21.15.71:0/1351101738 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f55cc0ad990 con 0x7f55d40ab950 2024-04-16T15:41:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.231+0000 7f55d3fff640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f55b406c920 0x7f55b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.232+0000 7f55d3fff640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f55b406c920 0x7f55b406ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f55d41557c0 tx=0x7f55bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:30.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.236+0000 7f55d17fa640 1 -- 172.21.15.71:0/1351101738 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55cc040070 con 0x7f55d40ab950 2024-04-16T15:41:30.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.408+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55d41dd6a0 con 0x7f55b406c920 2024-04-16T15:41:30.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.414+0000 7f55d17fa640 1 -- 172.21.15.71:0/1351101738 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f55d41dd6a0 con 0x7f55b406c920 2024-04-16T15:41:30.416 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:41:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f55b406c920 msgr2=0x7f55b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f55b406c920 0x7f55b406ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f55d41557c0 tx=0x7f55bc009040 comp rx=0 tx=0).stop 2024-04-16T15:41:30.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d40ab950 msgr2=0x7f55d414b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d40ab950 0x7f55d414b610 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f55cc000c00 tx=0x7f55cc0046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 shutdown_connections 2024-04-16T15:41:30.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f55b406c920 0x7f55b406ede0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55d414a230 0x7f55d4149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55d4150f40 0x7f55d41ed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 --2- 172.21.15.71:0/1351101738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d40ab950 0x7f55d414b610 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 >> 172.21.15.71:0/1351101738 conn(0x7f55d40c3e10 msgr2=0x7f55d414e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:30.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 shutdown_connections 2024-04-16T15:41:30.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.417+0000 7f55db255640 1 -- 172.21.15.71:0/1351101738 wait complete. 2024-04-16T15:41:30.428 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-04-16T15:41:29.713716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220816,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753904,"statfs":{"total":773094113280,"available":772867997696,"internally_reserved":0,"allocated":4759552,"data_stored":2707140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.209338"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530182+0000","last_change":"2024-04-16T15:41:23.523079+0000","last_active":"2024-04-16T15:41:24.530182+0000","last_peered":"2024-04-16T15:41:24.530182+0000","last_clean":"2024-04-16T15:41:24.530182+0000","last_became_active":"2024-04-16T15:41:23.522804+0000","last_became_peered":"2024-04-16T15:41:23.522804+0000","last_unstale":"2024-04-16T15:41:24.530182+0000","last_undegraded":"2024-04-16T15:41:24.530182+0000","last_fullsized":"2024-04-16T15:41:24.530182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530026+0000","last_change":"2024-04-16T15:41:23.518347+0000","last_active":"2024-04-16T15:41:24.530026+0000","last_peered":"2024-04-16T15:41:24.530026+0000","last_clean":"2024-04-16T15:41:24.530026+0000","last_became_active":"2024-04-16T15:41:23.517372+0000","last_became_peered":"2024-04-16T15:41:23.517372+0000","last_unstale":"2024-04-16T15:41:24.530026+0000","last_undegraded":"2024-04-16T15:41:24.530026+0000","last_fullsized":"2024-04-16T15:41:24.530026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.533697+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.533520+0000","last_became_peered":"2024-04-16T15:41:23.533520+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069345+0000","last_change":"2024-04-16T15:41:23.521655+0000","last_active":"2024-04-16T15:41:25.069345+0000","last_peered":"2024-04-16T15:41:25.069345+0000","last_clean":"2024-04-16T15:41:25.069345+0000","last_became_active":"2024-04-16T15:41:23.521419+0000","last_became_peered":"2024-04-16T15:41:23.521419+0000","last_unstale":"2024-04-16T15:41:25.069345+0000","last_undegraded":"2024-04-16T15:41:25.069345+0000","last_fullsized":"2024-04-16T15:41:25.069345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:48.607343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069423+0000","last_change":"2024-04-16T15:41:23.522246+0000","last_active":"2024-04-16T15:41:25.069423+0000","last_peered":"2024-04-16T15:41:25.069423+0000","last_clean":"2024-04-16T15:41:25.069423+0000","last_became_active":"2024-04-16T15:41:23.522009+0000","last_became_peered":"2024-04-16T15:41:23.522009+0000","last_unstale":"2024-04-16T15:41:25.069423+0000","last_undegraded":"2024-04-16T15:41:25.069423+0000","last_fullsized":"2024-04-16T15:41:25.069423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:33.728092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069091+0000","last_change":"2024-04-16T15:41:23.523004+0000","last_active":"2024-04-16T15:41:25.069091+0000","last_peered":"2024-04-16T15:41:25.069091+0000","last_clean":"2024-04-16T15:41:25.069091+0000","last_became_active":"2024-04-16T15:41:23.522742+0000","last_became_peered":"2024-04-16T15:41:23.522742+0000","last_unstale":"2024-04-16T15:41:25.069091+0000","last_undegraded":"2024-04-16T15:41:25.069091+0000","last_fullsized":"2024-04-16T15:41:25.069091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527561+0000","last_change":"2024-04-16T15:41:23.523305+0000","last_active":"2024-04-16T15:41:24.527561+0000","last_peered":"2024-04-16T15:41:24.527561+0000","last_clean":"2024-04-16T15:41:24.527561+0000","last_became_active":"2024-04-16T15:41:23.523022+0000","last_became_peered":"2024-04-16T15:41:23.523022+0000","last_unstale":"2024-04-16T15:41:24.527561+0000","last_undegraded":"2024-04-16T15:41:24.527561+0000","last_fullsized":"2024-04-16T15:41:24.527561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529666+0000","last_change":"2024-04-16T15:41:23.535446+0000","last_active":"2024-04-16T15:41:24.529666+0000","last_peered":"2024-04-16T15:41:24.529666+0000","last_clean":"2024-04-16T15:41:24.529666+0000","last_became_active":"2024-04-16T15:41:23.535156+0000","last_became_peered":"2024-04-16T15:41:23.535156+0000","last_unstale":"2024-04-16T15:41:24.529666+0000","last_undegraded":"2024-04-16T15:41:24.529666+0000","last_fullsized":"2024-04-16T15:41:24.529666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069784+0000","last_change":"2024-04-16T15:41:23.535019+0000","last_active":"2024-04-16T15:41:25.069784+0000","last_peered":"2024-04-16T15:41:25.069784+0000","last_clean":"2024-04-16T15:41:25.069784+0000","last_became_active":"2024-04-16T15:41:23.534726+0000","last_became_peered":"2024-04-16T15:41:23.534726+0000","last_unstale":"2024-04-16T15:41:25.069784+0000","last_undegraded":"2024-04-16T15:41:25.069784+0000","last_fullsized":"2024-04-16T15:41:25.069784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528288+0000","last_change":"2024-04-16T15:41:23.522709+0000","last_active":"2024-04-16T15:41:24.528288+0000","last_peered":"2024-04-16T15:41:24.528288+0000","last_clean":"2024-04-16T15:41:24.528288+0000","last_became_active":"2024-04-16T15:41:23.522437+0000","last_became_peered":"2024-04-16T15:41:23.522437+0000","last_unstale":"2024-04-16T15:41:24.528288+0000","last_undegraded":"2024-04-16T15:41:24.528288+0000","last_fullsized":"2024-04-16T15:41:24.528288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528764+0000","last_change":"2024-04-16T15:41:23.530620+0000","last_active":"2024-04-16T15:41:24.528764+0000","last_peered":"2024-04-16T15:41:24.528764+0000","last_clean":"2024-04-16T15:41:24.528764+0000","last_became_active":"2024-04-16T15:41:23.530320+0000","last_became_peered":"2024-04-16T15:41:23.530320+0000","last_unstale":"2024-04-16T15:41:24.528764+0000","last_undegraded":"2024-04-16T15:41:24.528764+0000","last_fullsized":"2024-04-16T15:41:24.528764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068927+0000","last_change":"2024-04-16T15:41:23.523419+0000","last_active":"2024-04-16T15:41:25.068927+0000","last_peered":"2024-04-16T15:41:25.068927+0000","last_clean":"2024-04-16T15:41:25.068927+0000","last_became_active":"2024-04-16T15:41:23.523081+0000","last_became_peered":"2024-04-16T15:41:23.523081+0000","last_unstale":"2024-04-16T15:41:25.068927+0000","last_undegraded":"2024-04-16T15:41:25.068927+0000","last_fullsized":"2024-04-16T15:41:25.068927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527828+0000","last_change":"2024-04-16T15:41:23.522227+0000","last_active":"2024-04-16T15:41:24.527828+0000","last_peered":"2024-04-16T15:41:24.527828+0000","last_clean":"2024-04-16T15:41:24.527828+0000","last_became_active":"2024-04-16T15:41:23.521531+0000","last_became_peered":"2024-04-16T15:41:23.521531+0000","last_unstale":"2024-04-16T15:41:24.527828+0000","last_undegraded":"2024-04-16T15:41:24.527828+0000","last_fullsized":"2024-04-16T15:41:24.527828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683427+0000","last_change":"2024-04-16T15:41:23.529710+0000","last_active":"2024-04-16T15:41:24.683427+0000","last_peered":"2024-04-16T15:41:24.683427+0000","last_clean":"2024-04-16T15:41:24.683427+0000","last_became_active":"2024-04-16T15:41:23.528557+0000","last_became_peered":"2024-04-16T15:41:23.528557+0000","last_unstale":"2024-04-16T15:41:24.683427+0000","last_undegraded":"2024-04-16T15:41:24.683427+0000","last_fullsized":"2024-04-16T15:41:24.683427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527689+0000","last_change":"2024-04-16T15:41:23.513641+0000","last_active":"2024-04-16T15:41:24.527689+0000","last_peered":"2024-04-16T15:41:24.527689+0000","last_clean":"2024-04-16T15:41:24.527689+0000","last_became_active":"2024-04-16T15:41:23.513340+0000","last_became_peered":"2024-04-16T15:41:23.513340+0000","last_unstale":"2024-04-16T15:41:24.527689+0000","last_undegraded":"2024-04-16T15:41:24.527689+0000","last_fullsized":"2024-04-16T15:41:24.527689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527345+0000","last_change":"2024-04-16T15:41:23.535782+0000","last_active":"2024-04-16T15:41:24.527345+0000","last_peered":"2024-04-16T15:41:24.527345+0000","last_clean":"2024-04-16T15:41:24.527345+0000","last_became_active":"2024-04-16T15:41:23.535549+0000","last_became_peered":"2024-04-16T15:41:23.535549+0000","last_unstale":"2024-04-16T15:41:24.527345+0000","last_undegraded":"2024-04-16T15:41:24.527345+0000","last_fullsized":"2024-04-16T15:41:24.527345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528422+0000","last_change":"2024-04-16T15:41:23.536402+0000","last_active":"2024-04-16T15:41:24.528422+0000","last_peered":"2024-04-16T15:41:24.528422+0000","last_clean":"2024-04-16T15:41:24.528422+0000","last_became_active":"2024-04-16T15:41:23.536009+0000","last_became_peered":"2024-04-16T15:41:23.536009+0000","last_unstale":"2024-04-16T15:41:24.528422+0000","last_undegraded":"2024-04-16T15:41:24.528422+0000","last_fullsized":"2024-04-16T15:41:24.528422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068843+0000","last_change":"2024-04-16T15:41:23.523701+0000","last_active":"2024-04-16T15:41:25.068843+0000","last_peered":"2024-04-16T15:41:25.068843+0000","last_clean":"2024-04-16T15:41:25.068843+0000","last_became_active":"2024-04-16T15:41:23.523442+0000","last_became_peered":"2024-04-16T15:41:23.523442+0000","last_unstale":"2024-04-16T15:41:25.068843+0000","last_undegraded":"2024-04-16T15:41:25.068843+0000","last_fullsized":"2024-04-16T15:41:25.068843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070293+0000","last_change":"2024-04-16T15:41:23.519553+0000","last_active":"2024-04-16T15:41:25.070293+0000","last_peered":"2024-04-16T15:41:25.070293+0000","last_clean":"2024-04-16T15:41:25.070293+0000","last_became_active":"2024-04-16T15:41:23.519258+0000","last_became_peered":"2024-04-16T15:41:23.519258+0000","last_unstale":"2024-04-16T15:41:25.070293+0000","last_undegraded":"2024-04-16T15:41:25.070293+0000","last_fullsized":"2024-04-16T15:41:25.070293+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:51.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068696+0000","last_change":"2024-04-16T15:41:23.522212+0000","last_active":"2024-04-16T15:41:25.068696+0000","last_peered":"2024-04-16T15:41:25.068696+0000","last_clean":"2024-04-16T15:41:25.068696+0000","last_became_active":"2024-04-16T15:41:23.521934+0000","last_became_peered":"2024-04-16T15:41:23.521934+0000","last_unstale":"2024-04-16T15:41:25.068696+0000","last_undegraded":"2024-04-16T15:41:25.068696+0000","last_fullsized":"2024-04-16T15:41:25.068696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:03.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527973+0000","last_change":"2024-04-16T15:41:23.520977+0000","last_active":"2024-04-16T15:41:24.527973+0000","last_peered":"2024-04-16T15:41:24.527973+0000","last_clean":"2024-04-16T15:41:24.527973+0000","last_became_active":"2024-04-16T15:41:23.520677+0000","last_became_peered":"2024-04-16T15:41:23.520677+0000","last_unstale":"2024-04-16T15:41:24.527973+0000","last_undegraded":"2024-04-16T15:41:24.527973+0000","last_fullsized":"2024-04-16T15:41:24.527973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683729+0000","last_change":"2024-04-16T15:41:23.527968+0000","last_active":"2024-04-16T15:41:24.683729+0000","last_peered":"2024-04-16T15:41:24.683729+0000","last_clean":"2024-04-16T15:41:24.683729+0000","last_became_active":"2024-04-16T15:41:23.527690+0000","last_became_peered":"2024-04-16T15:41:23.527690+0000","last_unstale":"2024-04-16T15:41:24.683729+0000","last_undegraded":"2024-04-16T15:41:24.683729+0000","last_fullsized":"2024-04-16T15:41:24.683729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527416+0000","last_change":"2024-04-16T15:41:23.527691+0000","last_active":"2024-04-16T15:41:24.527416+0000","last_peered":"2024-04-16T15:41:24.527416+0000","last_clean":"2024-04-16T15:41:24.527416+0000","last_became_active":"2024-04-16T15:41:23.527372+0000","last_became_peered":"2024-04-16T15:41:23.527372+0000","last_unstale":"2024-04-16T15:41:24.527416+0000","last_undegraded":"2024-04-16T15:41:24.527416+0000","last_fullsized":"2024-04-16T15:41:24.527416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069312+0000","last_change":"2024-04-16T15:41:23.525188+0000","last_active":"2024-04-16T15:41:25.069312+0000","last_peered":"2024-04-16T15:41:25.069312+0000","last_clean":"2024-04-16T15:41:25.069312+0000","last_became_active":"2024-04-16T15:41:23.524939+0000","last_became_peered":"2024-04-16T15:41:23.524939+0000","last_unstale":"2024-04-16T15:41:25.069312+0000","last_undegraded":"2024-04-16T15:41:25.069312+0000","last_fullsized":"2024-04-16T15:41:25.069312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:53.692430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068745+0000","last_change":"2024-04-16T15:41:23.526266+0000","last_active":"2024-04-16T15:41:25.068745+0000","last_peered":"2024-04-16T15:41:25.068745+0000","last_clean":"2024-04-16T15:41:25.068745+0000","last_became_active":"2024-04-16T15:41:23.525999+0000","last_became_peered":"2024-04-16T15:41:23.525999+0000","last_unstale":"2024-04-16T15:41:25.068745+0000","last_undegraded":"2024-04-16T15:41:25.068745+0000","last_fullsized":"2024-04-16T15:41:25.068745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:38.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069155+0000","last_change":"2024-04-16T15:41:23.535525+0000","last_active":"2024-04-16T15:41:25.069155+0000","last_peered":"2024-04-16T15:41:25.069155+0000","last_clean":"2024-04-16T15:41:25.069155+0000","last_became_active":"2024-04-16T15:41:23.535192+0000","last_became_peered":"2024-04-16T15:41:23.535192+0000","last_unstale":"2024-04-16T15:41:25.069155+0000","last_undegraded":"2024-04-16T15:41:25.069155+0000","last_fullsized":"2024-04-16T15:41:25.069155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068809+0000","last_change":"2024-04-16T15:41:23.527164+0000","last_active":"2024-04-16T15:41:25.068809+0000","last_peered":"2024-04-16T15:41:25.068809+0000","last_clean":"2024-04-16T15:41:25.068809+0000","last_became_active":"2024-04-16T15:41:23.526765+0000","last_became_peered":"2024-04-16T15:41:23.526765+0000","last_unstale":"2024-04-16T15:41:25.068809+0000","last_undegraded":"2024-04-16T15:41:25.068809+0000","last_fullsized":"2024-04-16T15:41:25.068809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528932+0000","last_change":"2024-04-16T15:41:23.529704+0000","last_active":"2024-04-16T15:41:24.528932+0000","last_peered":"2024-04-16T15:41:24.528932+0000","last_clean":"2024-04-16T15:41:24.528932+0000","last_became_active":"2024-04-16T15:41:23.525283+0000","last_became_peered":"2024-04-16T15:41:23.525283+0000","last_unstale":"2024-04-16T15:41:24.528932+0000","last_undegraded":"2024-04-16T15:41:24.528932+0000","last_fullsized":"2024-04-16T15:41:24.528932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068718+0000","last_change":"2024-04-16T15:41:23.523057+0000","last_active":"2024-04-16T15:41:25.068718+0000","last_peered":"2024-04-16T15:41:25.068718+0000","last_clean":"2024-04-16T15:41:25.068718+0000","last_became_active":"2024-04-16T15:41:23.522762+0000","last_became_peered":"2024-04-16T15:41:23.522762+0000","last_unstale":"2024-04-16T15:41:25.068718+0000","last_undegraded":"2024-04-16T15:41:25.068718+0000","last_fullsized":"2024-04-16T15:41:25.068718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:57.277773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068960+0000","last_change":"2024-04-16T15:41:23.522445+0000","last_active":"2024-04-16T15:41:25.068960+0000","last_peered":"2024-04-16T15:41:25.068960+0000","last_clean":"2024-04-16T15:41:25.068960+0000","last_became_active":"2024-04-16T15:41:23.522179+0000","last_became_peered":"2024-04-16T15:41:23.522179+0000","last_unstale":"2024-04-16T15:41:25.068960+0000","last_undegraded":"2024-04-16T15:41:25.068960+0000","last_fullsized":"2024-04-16T15:41:25.068960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527557+0000","last_change":"2024-04-16T15:41:23.528582+0000","last_active":"2024-04-16T15:41:24.527557+0000","last_peered":"2024-04-16T15:41:24.527557+0000","last_clean":"2024-04-16T15:41:24.527557+0000","last_became_active":"2024-04-16T15:41:23.528304+0000","last_became_peered":"2024-04-16T15:41:23.528304+0000","last_unstale":"2024-04-16T15:41:24.527557+0000","last_undegraded":"2024-04-16T15:41:24.527557+0000","last_fullsized":"2024-04-16T15:41:24.527557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:58:58.124408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069657+0000","last_change":"2024-04-16T15:41:23.519347+0000","last_active":"2024-04-16T15:41:25.069657+0000","last_peered":"2024-04-16T15:41:25.069657+0000","last_clean":"2024-04-16T15:41:25.069657+0000","last_became_active":"2024-04-16T15:41:23.519092+0000","last_became_peered":"2024-04-16T15:41:23.519092+0000","last_unstale":"2024-04-16T15:41:25.069657+0000","last_undegraded":"2024-04-16T15:41:25.069657+0000","last_fullsized":"2024-04-16T15:41:25.069657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527966+0000","last_change":"2024-04-16T15:41:23.536251+0000","last_active":"2024-04-16T15:41:24.527966+0000","last_peered":"2024-04-16T15:41:24.527966+0000","last_clean":"2024-04-16T15:41:24.527966+0000","last_became_active":"2024-04-16T15:41:23.535963+0000","last_became_peered":"2024-04-16T15:41:23.535963+0000","last_unstale":"2024-04-16T15:41:24.527966+0000","last_undegraded":"2024-04-16T15:41:24.527966+0000","last_fullsized":"2024-04-16T15:41:24.527966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068769+0000","last_change":"2024-04-16T15:41:22.522796+0000","last_active":"2024-04-16T15:41:25.068769+0000","last_peered":"2024-04-16T15:41:25.068769+0000","last_clean":"2024-04-16T15:41:25.068769+0000","last_became_active":"2024-04-16T15:41:22.522542+0000","last_became_peered":"2024-04-16T15:41:22.522542+0000","last_unstale":"2024-04-16T15:41:25.068769+0000","last_undegraded":"2024-04-16T15:41:25.068769+0000","last_fullsized":"2024-04-16T15:41:25.068769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069565+0000","last_change":"2024-04-16T15:41:23.536192+0000","last_active":"2024-04-16T15:41:25.069565+0000","last_peered":"2024-04-16T15:41:25.069565+0000","last_clean":"2024-04-16T15:41:25.069565+0000","last_became_active":"2024-04-16T15:41:23.535920+0000","last_became_peered":"2024-04-16T15:41:23.535920+0000","last_unstale":"2024-04-16T15:41:25.069565+0000","last_undegraded":"2024-04-16T15:41:25.069565+0000","last_fullsized":"2024-04-16T15:41:25.069565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069482+0000","last_change":"2024-04-16T15:41:22.523464+0000","last_active":"2024-04-16T15:41:25.069482+0000","last_peered":"2024-04-16T15:41:25.069482+0000","last_clean":"2024-04-16T15:41:25.069482+0000","last_became_active":"2024-04-16T15:41:22.523128+0000","last_became_peered":"2024-04-16T15:41:22.523128+0000","last_unstale":"2024-04-16T15:41:25.069482+0000","last_undegraded":"2024-04-16T15:41:25.069482+0000","last_fullsized":"2024-04-16T15:41:25.069482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683674+0000","last_change":"2024-04-16T15:41:23.530373+0000","last_active":"2024-04-16T15:41:24.683674+0000","last_peered":"2024-04-16T15:41:24.683674+0000","last_clean":"2024-04-16T15:41:24.683674+0000","last_became_active":"2024-04-16T15:41:23.530115+0000","last_became_peered":"2024-04-16T15:41:23.530115+0000","last_unstale":"2024-04-16T15:41:24.683674+0000","last_undegraded":"2024-04-16T15:41:24.683674+0000","last_fullsized":"2024-04-16T15:41:24.683674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069538+0000","last_change":"2024-04-16T15:41:22.521948+0000","last_active":"2024-04-16T15:41:25.069538+0000","last_peered":"2024-04-16T15:41:25.069538+0000","last_clean":"2024-04-16T15:41:25.069538+0000","last_became_active":"2024-04-16T15:41:22.521643+0000","last_became_peered":"2024-04-16T15:41:22.521643+0000","last_unstale":"2024-04-16T15:41:25.069538+0000","last_undegraded":"2024-04-16T15:41:25.069538+0000","last_fullsized":"2024-04-16T15:41:25.069538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530028+0000","last_change":"2024-04-16T15:41:23.528874+0000","last_active":"2024-04-16T15:41:24.530028+0000","last_peered":"2024-04-16T15:41:24.530028+0000","last_clean":"2024-04-16T15:41:24.530028+0000","last_became_active":"2024-04-16T15:41:23.528569+0000","last_became_peered":"2024-04-16T15:41:23.528569+0000","last_unstale":"2024-04-16T15:41:24.530028+0000","last_undegraded":"2024-04-16T15:41:24.530028+0000","last_fullsized":"2024-04-16T15:41:24.530028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683536+0000","last_change":"2024-04-16T15:41:22.525207+0000","last_active":"2024-04-16T15:41:24.683536+0000","last_peered":"2024-04-16T15:41:24.683536+0000","last_clean":"2024-04-16T15:41:24.683536+0000","last_became_active":"2024-04-16T15:41:22.524864+0000","last_became_peered":"2024-04-16T15:41:22.524864+0000","last_unstale":"2024-04-16T15:41:24.683536+0000","last_undegraded":"2024-04-16T15:41:24.683536+0000","last_fullsized":"2024-04-16T15:41:24.683536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068873+0000","last_change":"2024-04-16T15:41:23.523861+0000","last_active":"2024-04-16T15:41:25.068873+0000","last_peered":"2024-04-16T15:41:25.068873+0000","last_clean":"2024-04-16T15:41:25.068873+0000","last_became_active":"2024-04-16T15:41:23.523587+0000","last_became_peered":"2024-04-16T15:41:23.523587+0000","last_unstale":"2024-04-16T15:41:25.068873+0000","last_undegraded":"2024-04-16T15:41:25.068873+0000","last_fullsized":"2024-04-16T15:41:25.068873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:22.526265+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:22.525849+0000","last_became_peered":"2024-04-16T15:41:22.525849+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:02.400517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528806+0000","last_change":"2024-04-16T15:41:23.521523+0000","last_active":"2024-04-16T15:41:24.528806+0000","last_peered":"2024-04-16T15:41:24.528806+0000","last_clean":"2024-04-16T15:41:24.528806+0000","last_became_active":"2024-04-16T15:41:23.520768+0000","last_became_peered":"2024-04-16T15:41:23.520768+0000","last_unstale":"2024-04-16T15:41:24.528806+0000","last_undegraded":"2024-04-16T15:41:24.528806+0000","last_fullsized":"2024-04-16T15:41:24.528806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527590+0000","last_change":"2024-04-16T15:41:22.528402+0000","last_active":"2024-04-16T15:41:24.527590+0000","last_peered":"2024-04-16T15:41:24.527590+0000","last_clean":"2024-04-16T15:41:24.527590+0000","last_became_active":"2024-04-16T15:41:22.527999+0000","last_became_peered":"2024-04-16T15:41:22.527999+0000","last_unstale":"2024-04-16T15:41:24.527590+0000","last_undegraded":"2024-04-16T15:41:24.527590+0000","last_fullsized":"2024-04-16T15:41:24.527590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683644+0000","last_change":"2024-04-16T15:41:23.535673+0000","last_active":"2024-04-16T15:41:24.683644+0000","last_peered":"2024-04-16T15:41:24.683644+0000","last_clean":"2024-04-16T15:41:24.683644+0000","last_became_active":"2024-04-16T15:41:23.535387+0000","last_became_peered":"2024-04-16T15:41:23.535387+0000","last_unstale":"2024-04-16T15:41:24.683644+0000","last_undegraded":"2024-04-16T15:41:24.683644+0000","last_fullsized":"2024-04-16T15:41:24.683644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070111+0000","last_change":"2024-04-16T15:41:22.526184+0000","last_active":"2024-04-16T15:41:25.070111+0000","last_peered":"2024-04-16T15:41:25.070111+0000","last_clean":"2024-04-16T15:41:25.070111+0000","last_became_active":"2024-04-16T15:41:22.525819+0000","last_became_peered":"2024-04-16T15:41:22.525819+0000","last_unstale":"2024-04-16T15:41:25.070111+0000","last_undegraded":"2024-04-16T15:41:25.070111+0000","last_fullsized":"2024-04-16T15:41:25.070111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530001+0000","last_change":"2024-04-16T15:41:23.522193+0000","last_active":"2024-04-16T15:41:24.530001+0000","last_peered":"2024-04-16T15:41:24.530001+0000","last_clean":"2024-04-16T15:41:24.530001+0000","last_became_active":"2024-04-16T15:41:23.521900+0000","last_became_peered":"2024-04-16T15:41:23.521900+0000","last_unstale":"2024-04-16T15:41:24.530001+0000","last_undegraded":"2024-04-16T15:41:24.530001+0000","last_fullsized":"2024-04-16T15:41:24.530001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.521519+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.521234+0000","last_became_peered":"2024-04-16T15:41:22.521234+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.521070+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.520819+0000","last_became_peered":"2024-04-16T15:41:23.520819+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:03:41.341706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530092+0000","last_change":"2024-04-16T15:41:22.519163+0000","last_active":"2024-04-16T15:41:24.530092+0000","last_peered":"2024-04-16T15:41:24.530092+0000","last_clean":"2024-04-16T15:41:24.530092+0000","last_became_active":"2024-04-16T15:41:22.518905+0000","last_became_peered":"2024-04-16T15:41:22.518905+0000","last_unstale":"2024-04-16T15:41:24.530092+0000","last_undegraded":"2024-04-16T15:41:24.530092+0000","last_fullsized":"2024-04-16T15:41:24.530092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:03.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528974+0000","last_change":"2024-04-16T15:41:23.528995+0000","last_active":"2024-04-16T15:41:24.528974+0000","last_peered":"2024-04-16T15:41:24.528974+0000","last_clean":"2024-04-16T15:41:24.528974+0000","last_became_active":"2024-04-16T15:41:23.526288+0000","last_became_peered":"2024-04-16T15:41:23.526288+0000","last_unstale":"2024-04-16T15:41:24.528974+0000","last_undegraded":"2024-04-16T15:41:24.528974+0000","last_fullsized":"2024-04-16T15:41:24.528974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528459+0000","last_change":"2024-04-16T15:41:22.525433+0000","last_active":"2024-04-16T15:41:24.528459+0000","last_peered":"2024-04-16T15:41:24.528459+0000","last_clean":"2024-04-16T15:41:24.528459+0000","last_became_active":"2024-04-16T15:41:22.524567+0000","last_became_peered":"2024-04-16T15:41:22.524567+0000","last_unstale":"2024-04-16T15:41:24.528459+0000","last_undegraded":"2024-04-16T15:41:24.528459+0000","last_fullsized":"2024-04-16T15:41:24.528459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:56.272300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069430+0000","last_change":"2024-04-16T15:41:23.522367+0000","last_active":"2024-04-16T15:41:25.069430+0000","last_peered":"2024-04-16T15:41:25.069430+0000","last_clean":"2024-04-16T15:41:25.069430+0000","last_became_active":"2024-04-16T15:41:23.521984+0000","last_became_peered":"2024-04-16T15:41:23.521984+0000","last_unstale":"2024-04-16T15:41:25.069430+0000","last_undegraded":"2024-04-16T15:41:25.069430+0000","last_fullsized":"2024-04-16T15:41:25.069430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528812+0000","last_change":"2024-04-16T15:41:22.523086+0000","last_active":"2024-04-16T15:41:24.528812+0000","last_peered":"2024-04-16T15:41:24.528812+0000","last_clean":"2024-04-16T15:41:24.528812+0000","last_became_active":"2024-04-16T15:41:22.522759+0000","last_became_peered":"2024-04-16T15:41:22.522759+0000","last_unstale":"2024-04-16T15:41:24.528812+0000","last_undegraded":"2024-04-16T15:41:24.528812+0000","last_fullsized":"2024-04-16T15:41:24.528812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683778+0000","last_change":"2024-04-16T15:41:23.529819+0000","last_active":"2024-04-16T15:41:24.683778+0000","last_peered":"2024-04-16T15:41:24.683778+0000","last_clean":"2024-04-16T15:41:24.683778+0000","last_became_active":"2024-04-16T15:41:23.529543+0000","last_became_peered":"2024-04-16T15:41:23.529543+0000","last_unstale":"2024-04-16T15:41:24.683778+0000","last_undegraded":"2024-04-16T15:41:24.683778+0000","last_fullsized":"2024-04-16T15:41:24.683778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:22.522411+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:22.522130+0000","last_became_peered":"2024-04-16T15:41:22.522130+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:41.323112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069849+0000","last_change":"2024-04-16T15:41:23.535122+0000","last_active":"2024-04-16T15:41:25.069849+0000","last_peered":"2024-04-16T15:41:25.069849+0000","last_clean":"2024-04-16T15:41:25.069849+0000","last_became_active":"2024-04-16T15:41:23.534825+0000","last_became_peered":"2024-04-16T15:41:23.534825+0000","last_unstale":"2024-04-16T15:41:25.069849+0000","last_undegraded":"2024-04-16T15:41:25.069849+0000","last_fullsized":"2024-04-16T15:41:25.069849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530105+0000","last_change":"2024-04-16T15:41:22.520252+0000","last_active":"2024-04-16T15:41:24.530105+0000","last_peered":"2024-04-16T15:41:24.530105+0000","last_clean":"2024-04-16T15:41:24.530105+0000","last_became_active":"2024-04-16T15:41:22.519829+0000","last_became_peered":"2024-04-16T15:41:22.519829+0000","last_unstale":"2024-04-16T15:41:24.530105+0000","last_undegraded":"2024-04-16T15:41:24.530105+0000","last_fullsized":"2024-04-16T15:41:24.530105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529732+0000","last_change":"2024-04-16T15:41:23.522432+0000","last_active":"2024-04-16T15:41:24.529732+0000","last_peered":"2024-04-16T15:41:24.529732+0000","last_clean":"2024-04-16T15:41:24.529732+0000","last_became_active":"2024-04-16T15:41:23.522159+0000","last_became_peered":"2024-04-16T15:41:23.522159+0000","last_unstale":"2024-04-16T15:41:24.529732+0000","last_undegraded":"2024-04-16T15:41:24.529732+0000","last_fullsized":"2024-04-16T15:41:24.529732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069145+0000","last_change":"2024-04-16T15:41:22.527064+0000","last_active":"2024-04-16T15:41:25.069145+0000","last_peered":"2024-04-16T15:41:25.069145+0000","last_clean":"2024-04-16T15:41:25.069145+0000","last_became_active":"2024-04-16T15:41:22.525811+0000","last_became_peered":"2024-04-16T15:41:22.525811+0000","last_unstale":"2024-04-16T15:41:25.069145+0000","last_undegraded":"2024-04-16T15:41:25.069145+0000","last_fullsized":"2024-04-16T15:41:25.069145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527532+0000","last_change":"2024-04-16T15:41:23.527859+0000","last_active":"2024-04-16T15:41:24.527532+0000","last_peered":"2024-04-16T15:41:24.527532+0000","last_clean":"2024-04-16T15:41:24.527532+0000","last_became_active":"2024-04-16T15:41:23.527295+0000","last_became_peered":"2024-04-16T15:41:23.527295+0000","last_unstale":"2024-04-16T15:41:24.527532+0000","last_undegraded":"2024-04-16T15:41:24.527532+0000","last_fullsized":"2024-04-16T15:41:24.527532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:00:18.934742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068514+0000","last_change":"2024-04-16T15:41:22.524475+0000","last_active":"2024-04-16T15:41:25.068514+0000","last_peered":"2024-04-16T15:41:25.068514+0000","last_clean":"2024-04-16T15:41:25.068514+0000","last_became_active":"2024-04-16T15:41:22.523303+0000","last_became_peered":"2024-04-16T15:41:22.523303+0000","last_unstale":"2024-04-16T15:41:25.068514+0000","last_undegraded":"2024-04-16T15:41:25.068514+0000","last_fullsized":"2024-04-16T15:41:25.068514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:23.523183+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:23.522845+0000","last_became_peered":"2024-04-16T15:41:23.522845+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527394+0000","last_change":"2024-04-16T15:41:22.522941+0000","last_active":"2024-04-16T15:41:24.527394+0000","last_peered":"2024-04-16T15:41:24.527394+0000","last_clean":"2024-04-16T15:41:24.527394+0000","last_became_active":"2024-04-16T15:41:22.522633+0000","last_became_peered":"2024-04-16T15:41:22.522633+0000","last_unstale":"2024-04-16T15:41:24.527394+0000","last_undegraded":"2024-04-16T15:41:24.527394+0000","last_fullsized":"2024-04-16T15:41:24.527394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529991+0000","last_change":"2024-04-16T15:41:23.528140+0000","last_active":"2024-04-16T15:41:24.529991+0000","last_peered":"2024-04-16T15:41:24.529991+0000","last_clean":"2024-04-16T15:41:24.529991+0000","last_became_active":"2024-04-16T15:41:23.527841+0000","last_became_peered":"2024-04-16T15:41:23.527841+0000","last_unstale":"2024-04-16T15:41:24.529991+0000","last_undegraded":"2024-04-16T15:41:24.529991+0000","last_fullsized":"2024-04-16T15:41:24.529991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528233+0000","last_change":"2024-04-16T15:41:22.526961+0000","last_active":"2024-04-16T15:41:24.528233+0000","last_peered":"2024-04-16T15:41:24.528233+0000","last_clean":"2024-04-16T15:41:24.528233+0000","last_became_active":"2024-04-16T15:41:22.526140+0000","last_became_peered":"2024-04-16T15:41:22.526140+0000","last_unstale":"2024-04-16T15:41:24.528233+0000","last_undegraded":"2024-04-16T15:41:24.528233+0000","last_fullsized":"2024-04-16T15:41:24.528233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528839+0000","last_change":"2024-04-16T15:41:23.521298+0000","last_active":"2024-04-16T15:41:24.528839+0000","last_peered":"2024-04-16T15:41:24.528839+0000","last_clean":"2024-04-16T15:41:24.528839+0000","last_became_active":"2024-04-16T15:41:23.521041+0000","last_became_peered":"2024-04-16T15:41:23.521041+0000","last_unstale":"2024-04-16T15:41:24.528839+0000","last_undegraded":"2024-04-16T15:41:24.528839+0000","last_fullsized":"2024-04-16T15:41:24.528839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683715+0000","last_change":"2024-04-16T15:41:22.529046+0000","last_active":"2024-04-16T15:41:24.683715+0000","last_peered":"2024-04-16T15:41:24.683715+0000","last_clean":"2024-04-16T15:41:24.683715+0000","last_became_active":"2024-04-16T15:41:22.528745+0000","last_became_peered":"2024-04-16T15:41:22.528745+0000","last_unstale":"2024-04-16T15:41:24.683715+0000","last_undegraded":"2024-04-16T15:41:24.683715+0000","last_fullsized":"2024-04-16T15:41:24.683715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683671+0000","last_change":"2024-04-16T15:41:23.522758+0000","last_active":"2024-04-16T15:41:24.683671+0000","last_peered":"2024-04-16T15:41:24.683671+0000","last_clean":"2024-04-16T15:41:24.683671+0000","last_became_active":"2024-04-16T15:41:23.521964+0000","last_became_peered":"2024-04-16T15:41:23.521964+0000","last_unstale":"2024-04-16T15:41:24.683671+0000","last_undegraded":"2024-04-16T15:41:24.683671+0000","last_fullsized":"2024-04-16T15:41:24.683671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070256+0000","last_change":"2024-04-16T15:41:22.523228+0000","last_active":"2024-04-16T15:41:25.070256+0000","last_peered":"2024-04-16T15:41:25.070256+0000","last_clean":"2024-04-16T15:41:25.070256+0000","last_became_active":"2024-04-16T15:41:22.522888+0000","last_became_peered":"2024-04-16T15:41:22.522888+0000","last_unstale":"2024-04-16T15:41:25.070256+0000","last_undegraded":"2024-04-16T15:41:25.070256+0000","last_fullsized":"2024-04-16T15:41:25.070256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:59.195492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069536+0000","last_change":"2024-04-16T15:41:23.525797+0000","last_active":"2024-04-16T15:41:25.069536+0000","last_peered":"2024-04-16T15:41:25.069536+0000","last_clean":"2024-04-16T15:41:25.069536+0000","last_became_active":"2024-04-16T15:41:23.525537+0000","last_became_peered":"2024-04-16T15:41:23.525537+0000","last_unstale":"2024-04-16T15:41:25.069536+0000","last_undegraded":"2024-04-16T15:41:25.069536+0000","last_fullsized":"2024-04-16T15:41:25.069536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:06.161198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068884+0000","last_change":"2024-04-16T15:41:22.521885+0000","last_active":"2024-04-16T15:41:25.068884+0000","last_peered":"2024-04-16T15:41:25.068884+0000","last_clean":"2024-04-16T15:41:25.068884+0000","last_became_active":"2024-04-16T15:41:22.521574+0000","last_became_peered":"2024-04-16T15:41:22.521574+0000","last_unstale":"2024-04-16T15:41:25.068884+0000","last_undegraded":"2024-04-16T15:41:25.068884+0000","last_fullsized":"2024-04-16T15:41:25.068884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527576+0000","last_change":"2024-04-16T15:41:23.529208+0000","last_active":"2024-04-16T15:41:24.527576+0000","last_peered":"2024-04-16T15:41:24.527576+0000","last_clean":"2024-04-16T15:41:24.527576+0000","last_became_active":"2024-04-16T15:41:23.528946+0000","last_became_peered":"2024-04-16T15:41:23.528946+0000","last_unstale":"2024-04-16T15:41:24.527576+0000","last_undegraded":"2024-04-16T15:41:24.527576+0000","last_fullsized":"2024-04-16T15:41:24.527576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:59:03.633363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069471+0000","last_change":"2024-04-16T15:41:22.525515+0000","last_active":"2024-04-16T15:41:25.069471+0000","last_peered":"2024-04-16T15:41:25.069471+0000","last_clean":"2024-04-16T15:41:25.069471+0000","last_became_active":"2024-04-16T15:41:22.525228+0000","last_became_peered":"2024-04-16T15:41:22.525228+0000","last_unstale":"2024-04-16T15:41:25.069471+0000","last_undegraded":"2024-04-16T15:41:25.069471+0000","last_fullsized":"2024-04-16T15:41:25.069471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:55:38.421964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068802+0000","last_change":"2024-04-16T15:41:23.522196+0000","last_active":"2024-04-16T15:41:25.068802+0000","last_peered":"2024-04-16T15:41:25.068802+0000","last_clean":"2024-04-16T15:41:25.068802+0000","last_became_active":"2024-04-16T15:41:23.521931+0000","last_became_peered":"2024-04-16T15:41:23.521931+0000","last_unstale":"2024-04-16T15:41:25.068802+0000","last_undegraded":"2024-04-16T15:41:25.068802+0000","last_fullsized":"2024-04-16T15:41:25.068802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069160+0000","last_change":"2024-04-16T15:41:22.523743+0000","last_active":"2024-04-16T15:41:25.069160+0000","last_peered":"2024-04-16T15:41:25.069160+0000","last_clean":"2024-04-16T15:41:25.069160+0000","last_became_active":"2024-04-16T15:41:22.523348+0000","last_became_peered":"2024-04-16T15:41:22.523348+0000","last_unstale":"2024-04-16T15:41:25.069160+0000","last_undegraded":"2024-04-16T15:41:25.069160+0000","last_fullsized":"2024-04-16T15:41:25.069160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:48.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528766+0000","last_change":"2024-04-16T15:41:23.524838+0000","last_active":"2024-04-16T15:41:24.528766+0000","last_peered":"2024-04-16T15:41:24.528766+0000","last_clean":"2024-04-16T15:41:24.528766+0000","last_became_active":"2024-04-16T15:41:23.524540+0000","last_became_peered":"2024-04-16T15:41:23.524540+0000","last_unstale":"2024-04-16T15:41:24.528766+0000","last_undegraded":"2024-04-16T15:41:24.528766+0000","last_fullsized":"2024-04-16T15:41:24.528766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527606+0000","last_change":"2024-04-16T15:41:22.522239+0000","last_active":"2024-04-16T15:41:24.527606+0000","last_peered":"2024-04-16T15:41:24.527606+0000","last_clean":"2024-04-16T15:41:24.527606+0000","last_became_active":"2024-04-16T15:41:22.521860+0000","last_became_peered":"2024-04-16T15:41:22.521860+0000","last_unstale":"2024-04-16T15:41:24.527606+0000","last_undegraded":"2024-04-16T15:41:24.527606+0000","last_fullsized":"2024-04-16T15:41:24.527606+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530187+0000","last_change":"2024-04-16T15:41:23.522134+0000","last_active":"2024-04-16T15:41:24.530187+0000","last_peered":"2024-04-16T15:41:24.530187+0000","last_clean":"2024-04-16T15:41:24.530187+0000","last_became_active":"2024-04-16T15:41:23.521852+0000","last_became_peered":"2024-04-16T15:41:23.521852+0000","last_unstale":"2024-04-16T15:41:24.530187+0000","last_undegraded":"2024-04-16T15:41:24.530187+0000","last_fullsized":"2024-04-16T15:41:24.530187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069255+0000","last_change":"2024-04-16T15:41:22.525713+0000","last_active":"2024-04-16T15:41:25.069255+0000","last_peered":"2024-04-16T15:41:25.069255+0000","last_clean":"2024-04-16T15:41:25.069255+0000","last_became_active":"2024-04-16T15:41:22.525423+0000","last_became_peered":"2024-04-16T15:41:22.525423+0000","last_unstale":"2024-04-16T15:41:25.069255+0000","last_undegraded":"2024-04-16T15:41:25.069255+0000","last_fullsized":"2024-04-16T15:41:25.069255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:22.255097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068828+0000","last_change":"2024-04-16T15:41:23.536118+0000","last_active":"2024-04-16T15:41:25.068828+0000","last_peered":"2024-04-16T15:41:25.068828+0000","last_clean":"2024-04-16T15:41:25.068828+0000","last_became_active":"2024-04-16T15:41:23.535863+0000","last_became_peered":"2024-04-16T15:41:23.535863+0000","last_unstale":"2024-04-16T15:41:25.068828+0000","last_undegraded":"2024-04-16T15:41:25.068828+0000","last_fullsized":"2024-04-16T15:41:25.068828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528405+0000","last_change":"2024-04-16T15:41:22.525073+0000","last_active":"2024-04-16T15:41:24.528405+0000","last_peered":"2024-04-16T15:41:24.528405+0000","last_clean":"2024-04-16T15:41:24.528405+0000","last_became_active":"2024-04-16T15:41:22.524464+0000","last_became_peered":"2024-04-16T15:41:22.524464+0000","last_unstale":"2024-04-16T15:41:24.528405+0000","last_undegraded":"2024-04-16T15:41:24.528405+0000","last_fullsized":"2024-04-16T15:41:24.528405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529022+0000","last_change":"2024-04-16T15:41:23.526852+0000","last_active":"2024-04-16T15:41:24.529022+0000","last_peered":"2024-04-16T15:41:24.529022+0000","last_clean":"2024-04-16T15:41:24.529022+0000","last_became_active":"2024-04-16T15:41:23.526580+0000","last_became_peered":"2024-04-16T15:41:23.526580+0000","last_unstale":"2024-04-16T15:41:24.529022+0000","last_undegraded":"2024-04-16T15:41:24.529022+0000","last_fullsized":"2024-04-16T15:41:24.529022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070264+0000","last_change":"2024-04-16T15:41:22.523803+0000","last_active":"2024-04-16T15:41:25.070264+0000","last_peered":"2024-04-16T15:41:25.070264+0000","last_clean":"2024-04-16T15:41:25.070264+0000","last_became_active":"2024-04-16T15:41:22.523524+0000","last_became_peered":"2024-04-16T15:41:22.523524+0000","last_unstale":"2024-04-16T15:41:25.070264+0000","last_undegraded":"2024-04-16T15:41:25.070264+0000","last_fullsized":"2024-04-16T15:41:25.070264+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:46.457169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530098+0000","last_change":"2024-04-16T15:41:23.521456+0000","last_active":"2024-04-16T15:41:24.530098+0000","last_peered":"2024-04-16T15:41:24.530098+0000","last_clean":"2024-04-16T15:41:24.530098+0000","last_became_active":"2024-04-16T15:41:23.521114+0000","last_became_peered":"2024-04-16T15:41:23.521114+0000","last_unstale":"2024-04-16T15:41:24.530098+0000","last_undegraded":"2024-04-16T15:41:24.530098+0000","last_fullsized":"2024-04-16T15:41:24.530098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069158+0000","last_change":"2024-04-16T15:41:22.517077+0000","last_active":"2024-04-16T15:41:25.069158+0000","last_peered":"2024-04-16T15:41:25.069158+0000","last_clean":"2024-04-16T15:41:25.069158+0000","last_became_active":"2024-04-16T15:41:22.516776+0000","last_became_peered":"2024-04-16T15:41:22.516776+0000","last_unstale":"2024-04-16T15:41:25.069158+0000","last_undegraded":"2024-04-16T15:41:25.069158+0000","last_fullsized":"2024-04-16T15:41:25.069158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070000+0000","last_change":"2024-04-16T15:41:23.521606+0000","last_active":"2024-04-16T15:41:25.070000+0000","last_peered":"2024-04-16T15:41:25.070000+0000","last_clean":"2024-04-16T15:41:25.070000+0000","last_became_active":"2024-04-16T15:41:23.521351+0000","last_became_peered":"2024-04-16T15:41:23.521351+0000","last_unstale":"2024-04-16T15:41:25.070000+0000","last_undegraded":"2024-04-16T15:41:25.070000+0000","last_fullsized":"2024-04-16T15:41:25.070000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069018+0000","last_change":"2024-04-16T15:41:22.517838+0000","last_active":"2024-04-16T15:41:25.069018+0000","last_peered":"2024-04-16T15:41:25.069018+0000","last_clean":"2024-04-16T15:41:25.069018+0000","last_became_active":"2024-04-16T15:41:22.517494+0000","last_became_peered":"2024-04-16T15:41:22.517494+0000","last_unstale":"2024-04-16T15:41:25.069018+0000","last_undegraded":"2024-04-16T15:41:25.069018+0000","last_fullsized":"2024-04-16T15:41:25.069018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:50.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070060+0000","last_change":"2024-04-16T15:41:23.524122+0000","last_active":"2024-04-16T15:41:25.070060+0000","last_peered":"2024-04-16T15:41:25.070060+0000","last_clean":"2024-04-16T15:41:25.070060+0000","last_became_active":"2024-04-16T15:41:23.523821+0000","last_became_peered":"2024-04-16T15:41:23.523821+0000","last_unstale":"2024-04-16T15:41:25.070060+0000","last_undegraded":"2024-04-16T15:41:25.070060+0000","last_fullsized":"2024-04-16T15:41:25.070060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068589+0000","last_change":"2024-04-16T15:41:22.519275+0000","last_active":"2024-04-16T15:41:25.068589+0000","last_peered":"2024-04-16T15:41:25.068589+0000","last_clean":"2024-04-16T15:41:25.068589+0000","last_became_active":"2024-04-16T15:41:22.518999+0000","last_became_peered":"2024-04-16T15:41:22.518999+0000","last_unstale":"2024-04-16T15:41:25.068589+0000","last_undegraded":"2024-04-16T15:41:25.068589+0000","last_fullsized":"2024-04-16T15:41:25.068589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683431+0000","last_change":"2024-04-16T15:41:23.522582+0000","last_active":"2024-04-16T15:41:24.683431+0000","last_peered":"2024-04-16T15:41:24.683431+0000","last_clean":"2024-04-16T15:41:24.683431+0000","last_became_active":"2024-04-16T15:41:23.522254+0000","last_became_peered":"2024-04-16T15:41:23.522254+0000","last_unstale":"2024-04-16T15:41:24.683431+0000","last_undegraded":"2024-04-16T15:41:24.683431+0000","last_fullsized":"2024-04-16T15:41:24.683431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.525440+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.523261+0000","last_became_peered":"2024-04-16T15:41:22.523261+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:23.526801+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:23.526499+0000","last_became_peered":"2024-04-16T15:41:23.526499+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529584+0000","last_change":"2024-04-16T15:41:22.521239+0000","last_active":"2024-04-16T15:41:24.529584+0000","last_peered":"2024-04-16T15:41:24.529584+0000","last_clean":"2024-04-16T15:41:24.529584+0000","last_became_active":"2024-04-16T15:41:22.520944+0000","last_became_peered":"2024-04-16T15:41:22.520944+0000","last_unstale":"2024-04-16T15:41:24.529584+0000","last_undegraded":"2024-04-16T15:41:24.529584+0000","last_fullsized":"2024-04-16T15:41:24.529584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528112+0000","last_change":"2024-04-16T15:41:23.532440+0000","last_active":"2024-04-16T15:41:24.528112+0000","last_peered":"2024-04-16T15:41:24.528112+0000","last_clean":"2024-04-16T15:41:24.528112+0000","last_became_active":"2024-04-16T15:41:23.530755+0000","last_became_peered":"2024-04-16T15:41:23.530755+0000","last_unstale":"2024-04-16T15:41:24.528112+0000","last_undegraded":"2024-04-16T15:41:24.528112+0000","last_fullsized":"2024-04-16T15:41:24.528112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528231+0000","last_change":"2024-04-16T15:41:22.523523+0000","last_active":"2024-04-16T15:41:24.528231+0000","last_peered":"2024-04-16T15:41:24.528231+0000","last_clean":"2024-04-16T15:41:24.528231+0000","last_became_active":"2024-04-16T15:41:22.522766+0000","last_became_peered":"2024-04-16T15:41:22.522766+0000","last_unstale":"2024-04-16T15:41:24.528231+0000","last_undegraded":"2024-04-16T15:41:24.528231+0000","last_fullsized":"2024-04-16T15:41:24.528231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527623+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:41:24.527623+0000","last_peered":"2024-04-16T15:41:24.527623+0000","last_clean":"2024-04-16T15:41:24.527623+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:41:24.527623+0000","last_undegraded":"2024-04-16T15:41:24.527623+0000","last_fullsized":"2024-04-16T15:41:24.527623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":191844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":780600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:41:30.432 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T15:41:30.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.575+0000 7f416c5b5640 1 Processor -- start 2024-04-16T15:41:30.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.575+0000 7f416c5b5640 1 -- start start 2024-04-16T15:41:30.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416c5b5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f41640c3f40 0x7f4164072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416c5b5640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f41640c3170 0x7f416415f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416c5b5640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41640c5b00 con 0x7f416406cbb0 2024-04-16T15:41:30.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416c5b5640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f41640c6390 con 0x7f41640c3170 2024-04-16T15:41:30.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416c5b5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f41640c6670 con 0x7f41640c3f40 2024-04-16T15:41:30.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416ab52640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f41640c3170 0x7f416415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416ab52640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f41640c3170 0x7f416415f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53710/0 (socket says 172.21.15.71:53710) 2024-04-16T15:41:30.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416ab52640 1 -- 172.21.15.71:0/2905845522 learned_addr learned my addr 172.21.15.71:0/2905845522 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:30.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f4169b50640 1 --2- 172.21.15.71:0/2905845522 >> v2:172.21.15.122:3300/0 conn(0x7f41640c3f40 0x7f4164072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416ab52640 1 -- 172.21.15.71:0/2905845522 >> v2:172.21.15.122:3300/0 conn(0x7f41640c3f40 msgr2=0x7f4164072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:30.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416ab52640 1 --2- 172.21.15.71:0/2905845522 >> v2:172.21.15.122:3300/0 conn(0x7f41640c3f40 0x7f4164072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416ab52640 1 -- 172.21.15.71:0/2905845522 >> v1:172.21.15.122:6789/0 conn(0x7f416406cbb0 legacy=0x7f41640c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:30.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f416ab52640 1 -- 172.21.15.71:0/2905845522 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41640c5da0 con 0x7f41640c3170 2024-04-16T15:41:30.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.576+0000 7f4169b50640 1 --2- 172.21.15.71:0/2905845522 >> v2:172.21.15.122:3300/0 conn(0x7f41640c3f40 0x7f4164072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:30.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.577+0000 7f416ab52640 1 --2- 172.21.15.71:0/2905845522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3170 0x7f416415f980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f415c009e50 tx=0x7f415c02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6870fd7c02892cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:30.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.577+0000 7f416934f640 1 -- 172.21.15.71:0/2905845522 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c02c960 con 0x7f41640c3170 2024-04-16T15:41:30.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.577+0000 7f416934f640 1 -- 172.21.15.71:0/2905845522 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f415c02cb00 con 0x7f41640c3170 2024-04-16T15:41:30.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.577+0000 7f416934f640 1 -- 172.21.15.71:0/2905845522 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c0343f0 con 0x7f41640c3170 2024-04-16T15:41:30.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.577+0000 7f416c5b5640 1 -- 172.21.15.71:0/2905845522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3170 msgr2=0x7f416415f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.577+0000 7f416c5b5640 1 --2- 172.21.15.71:0/2905845522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3170 0x7f416415f980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f415c009e50 tx=0x7f415c02bdc0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.578+0000 7f416c5b5640 1 -- 172.21.15.71:0/2905845522 shutdown_connections 2024-04-16T15:41:30.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.578+0000 7f416c5b5640 1 --2- 172.21.15.71:0/2905845522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3170 0x7f416415f980 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.578+0000 7f416c5b5640 1 --2- 172.21.15.71:0/2905845522 >> v2:172.21.15.122:3300/0 conn(0x7f41640c3f40 0x7f4164072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.578+0000 7f416c5b5640 1 -- 172.21.15.71:0/2905845522 >> 172.21.15.71:0/2905845522 conn(0x7f41640c45d0 msgr2=0x7f41640c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:30.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.578+0000 7f416c5b5640 1 -- 172.21.15.71:0/2905845522 shutdown_connections 2024-04-16T15:41:30.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.578+0000 7f416c5b5640 1 -- 172.21.15.71:0/2905845522 wait complete. 2024-04-16T15:41:30.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416c5b5640 1 Processor -- start 2024-04-16T15:41:30.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416c5b5640 1 -- start start 2024-04-16T15:41:30.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416c5b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f416406cbb0 0x7f41640626b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416c5b5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41640c3170 0x7f41640aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416c5b5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3f40 0x7f41641ee930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416c5b5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f41640c35e0 con 0x7f41640c3f40 2024-04-16T15:41:30.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416c5b5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4164073060 con 0x7f416406cbb0 2024-04-16T15:41:30.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416c5b5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f41640c3760 con 0x7f41640c3170 2024-04-16T15:41:30.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416ab52640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3f40 0x7f41641ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416ab52640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3f40 0x7f41641ee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53720/0 (socket says 172.21.15.71:53720) 2024-04-16T15:41:30.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.579+0000 7f416ab52640 1 -- 172.21.15.71:0/1263033854 learned_addr learned my addr 172.21.15.71:0/1263033854 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:30.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f416ab52640 1 -- 172.21.15.71:0/1263033854 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41640c3170 msgr2=0x7f41640aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f416ab52640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41640c3170 0x7f41640aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f416ab52640 1 -- 172.21.15.71:0/1263033854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f416406cbb0 msgr2=0x7f41640626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f416ab52640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f416406cbb0 0x7f41640626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f416ab52640 1 -- 172.21.15.71:0/1263033854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41640aae50 con 0x7f41640c3f40 2024-04-16T15:41:30.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f416ab52640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3f40 0x7f41641ee930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f415c009e50 tx=0x7f415c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:30.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f415affd640 1 -- 172.21.15.71:0/1263033854 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c043020 con 0x7f41640c3f40 2024-04-16T15:41:30.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f415affd640 1 -- 172.21.15.71:0/1263033854 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f415c03e450 con 0x7f41640c3f40 2024-04-16T15:41:30.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41640ab9c0 con 0x7f41640c3f40 2024-04-16T15:41:30.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f415affd640 1 -- 172.21.15.71:0/1263033854 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c03e820 con 0x7f41640c3f40 2024-04-16T15:41:30.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.580+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41640abf80 con 0x7f41640c3f40 2024-04-16T15:41:30.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.581+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4164068d20 con 0x7f41640c3f40 2024-04-16T15:41:30.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.581+0000 7f415affd640 1 -- 172.21.15.71:0/1263033854 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f415c046020 con 0x7f41640c3f40 2024-04-16T15:41:30.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.582+0000 7f415affd640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f414806c8f0 0x7f414806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:30.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.582+0000 7f415affd640 1 -- 172.21.15.71:0/1263033854 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f415c0ae5f0 con 0x7f41640c3f40 2024-04-16T15:41:30.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.582+0000 7f416a351640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f414806c8f0 0x7f414806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:30.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.582+0000 7f416a351640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f414806c8f0 0x7f414806edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4160003cc0 tx=0x7f41600079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:30.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.586+0000 7f415affd640 1 -- 172.21.15.71:0/1263033854 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f415c04d030 con 0x7f41640c3f40 2024-04-16T15:41:30.738 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-16T15:41:30.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.764+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41641e0630 con 0x7f41640c3f40 2024-04-16T15:41:30.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.765+0000 7f415affd640 1 -- 172.21.15.71:0/1263033854 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16930 (secure 0 0 0) 0x7f415c07c920 con 0x7f41640c3f40 2024-04-16T15:41:30.764 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:30.765 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"fsid":"c5697e20-5e32-4f13-b48e-2f7b8429c0b4","created":"2024-04-16T15:39:59.467673+0000","modified":"2024-04-16T15:41:24.513402+0000","last_up_change":"2024-04-16T15:40:10.753240+0000","last_in_change":"2024-04-16T15:40:01.774843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T15:40:11.711384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-16T15:41:21.360227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-16T15:41:21.816837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"76b3f430-8289-42d6-9cb4-93fba8f0da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6801","nonce":2582424538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6803","nonce":2582424538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6807","nonce":2582424538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2582424538},{"type":"v1","addr":"172.21.15.71:6805","nonce":2582424538}]},"public_addr":"172.21.15.71:6801/2582424538","cluster_addr":"172.21.15.71:6803/2582424538","heartbeat_back_addr":"172.21.15.71:6807/2582424538","heartbeat_front_addr":"172.21.15.71:6805/2582424538","state":["exists","up"]},{"osd":1,"uuid":"2e8ebb7f-6d73-4eea-8289-c4d0f5b7c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6809","nonce":357157073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6811","nonce":357157073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6818","nonce":357157073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":357157073},{"type":"v1","addr":"172.21.15.71:6814","nonce":357157073}]},"public_addr":"172.21.15.71:6809/357157073","cluster_addr":"172.21.15.71:6811/357157073","heartbeat_back_addr":"172.21.15.71:6818/357157073","heartbeat_front_addr":"172.21.15.71:6814/357157073","state":["exists","up"]},{"osd":2,"uuid":"2eec121c-f3b5-4f74-8b1e-f389c7d99f3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6813","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6815","nonce":3827429750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6817","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6819","nonce":3827429750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6823","nonce":3827429750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3827429750},{"type":"v1","addr":"172.21.15.71:6821","nonce":3827429750}]},"public_addr":"172.21.15.71:6815/3827429750","cluster_addr":"172.21.15.71:6819/3827429750","heartbeat_back_addr":"172.21.15.71:6823/3827429750","heartbeat_front_addr":"172.21.15.71:6821/3827429750","state":["exists","up"]},{"osd":3,"uuid":"f55ea5db-5519-4ae3-bcaa-1725a4eabd8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6825","nonce":2052589418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6827","nonce":2052589418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6831","nonce":2052589418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2052589418},{"type":"v1","addr":"172.21.15.71:6829","nonce":2052589418}]},"public_addr":"172.21.15.71:6825/2052589418","cluster_addr":"172.21.15.71:6827/2052589418","heartbeat_back_addr":"172.21.15.71:6831/2052589418","heartbeat_front_addr":"172.21.15.71:6829/2052589418","state":["exists","up"]},{"osd":4,"uuid":"adf7b0dd-31d3-483c-8914-b3efe320eccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6818","nonce":710391503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6822","nonce":710391503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6830","nonce":710391503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":710391503},{"type":"v1","addr":"172.21.15.122:6826","nonce":710391503}]},"public_addr":"172.21.15.122:6818/710391503","cluster_addr":"172.21.15.122:6822/710391503","heartbeat_back_addr":"172.21.15.122:6830/710391503","heartbeat_front_addr":"172.21.15.122:6826/710391503","state":["exists","up"]},{"osd":5,"uuid":"4669c4df-9ec7-40fe-96ae-49a6b0a97324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6817","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6819","nonce":1133463708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6823","nonce":1133463708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6829","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6831","nonce":1133463708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":1133463708},{"type":"v1","addr":"172.21.15.122:6827","nonce":1133463708}]},"public_addr":"172.21.15.122:6819/1133463708","cluster_addr":"172.21.15.122:6823/1133463708","heartbeat_back_addr":"172.21.15.122:6831/1133463708","heartbeat_front_addr":"172.21.15.122:6827/1133463708","state":["exists","up"]},{"osd":6,"uuid":"8defc38b-b99c-41f3-b1af-03ac9ac9eaa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6809","nonce":1958778321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6811","nonce":1958778321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6815","nonce":1958778321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1958778321},{"type":"v1","addr":"172.21.15.122:6813","nonce":1958778321}]},"public_addr":"172.21.15.122:6809/1958778321","cluster_addr":"172.21.15.122:6811/1958778321","heartbeat_back_addr":"172.21.15.122:6815/1958778321","heartbeat_front_addr":"172.21.15.122:6813/1958778321","state":["exists","up"]},{"osd":7,"uuid":"87c94baa-0e15-4613-93ff-4cf8d0a1db33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6801","nonce":3651959178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6803","nonce":3651959178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6807","nonce":3651959178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3651959178},{"type":"v1","addr":"172.21.15.122:6805","nonce":3651959178}]},"public_addr":"172.21.15.122:6801/3651959178","cluster_addr":"172.21.15.122:6803/3651959178","heartbeat_back_addr":"172.21.15.122:6807/3651959178","heartbeat_front_addr":"172.21.15.122:6805/3651959178","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.946922+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:07.007226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.951145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.945439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.941165+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.966066+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.899791+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T15:40:06.879319+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T15:41:30.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f414806c8f0 msgr2=0x7f414806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f414806c8f0 0x7f414806edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4160003cc0 tx=0x7f41600079b0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3f40 msgr2=0x7f41641ee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3f40 0x7f41641ee930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f415c009e50 tx=0x7f415c004a40 comp rx=0 tx=0).stop 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 shutdown_connections 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f414806c8f0 0x7f414806edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41640c3f40 0x7f41641ee930 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41640c3170 0x7f41640aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 --2- 172.21.15.71:0/1263033854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f416406cbb0 0x7f41640626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:30.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.767+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 >> 172.21.15.71:0/1263033854 conn(0x7f41640c45d0 msgr2=0x7f41641e0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:30.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.768+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 shutdown_connections 2024-04-16T15:41:30.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:30.768+0000 7f416c5b5640 1 -- 172.21.15.71:0/1263033854 wait complete. 2024-04-16T15:41:31.779 INFO:tasks.ceph:Scrubbing osd.0 2024-04-16T15:41:31.779 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-16T15:41:31.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.898+0000 7fd5578ba640 1 Processor -- start 2024-04-16T15:41:31.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.898+0000 7fd5578ba640 1 -- start start 2024-04-16T15:41:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd5578ba640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fd5500adb90 0x7fd5500c2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd5578ba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd55006f9a0 0x7fd55006fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd5578ba640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5500c2e70 con 0x7fd550070310 2024-04-16T15:41:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd5578ba640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fd5500adf70 con 0x7fd5500adb90 2024-04-16T15:41:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd5578ba640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fd550073450 con 0x7fd55006f9a0 2024-04-16T15:41:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd555656640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fd5500adb90 0x7fd5500c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd555656640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fd5500adb90 0x7fd5500c2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53730/0 (socket says 172.21.15.71:53730) 2024-04-16T15:41:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd555656640 1 -- 172.21.15.71:0/598998703 learned_addr learned my addr 172.21.15.71:0/598998703 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd555656640 1 -- 172.21.15.71:0/598998703 >> v2:172.21.15.122:3300/0 conn(0x7fd55006f9a0 msgr2=0x7fd55006fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd554e55640 1 --2- 172.21.15.71:0/598998703 >> v2:172.21.15.122:3300/0 conn(0x7fd55006f9a0 0x7fd55006fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd555656640 1 --2- 172.21.15.71:0/598998703 >> v2:172.21.15.122:3300/0 conn(0x7fd55006f9a0 0x7fd55006fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd555656640 1 -- 172.21.15.71:0/598998703 >> v1:172.21.15.122:6789/0 conn(0x7fd550070310 legacy=0x7fd550148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.899+0000 7fd555656640 1 -- 172.21.15.71:0/598998703 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5500c4d40 con 0x7fd5500adb90 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.900+0000 7fd555656640 1 --2- 172.21.15.71:0/598998703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5500adb90 0x7fd5500c2810 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd5500ab240 tx=0x7fd54c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b703fe21f4daa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.900+0000 7fd547fff640 1 -- 172.21.15.71:0/598998703 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c02e070 con 0x7fd5500adb90 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.900+0000 7fd547fff640 1 -- 172.21.15.71:0/598998703 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c037070 con 0x7fd5500adb90 2024-04-16T15:41:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.900+0000 7fd547fff640 1 -- 172.21.15.71:0/598998703 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c031a00 con 0x7fd5500adb90 2024-04-16T15:41:31.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.901+0000 7fd5578ba640 1 -- 172.21.15.71:0/598998703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5500adb90 msgr2=0x7fd5500c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:31.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.901+0000 7fd5578ba640 1 --2- 172.21.15.71:0/598998703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5500adb90 0x7fd5500c2810 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd5500ab240 tx=0x7fd54c02b880 comp rx=0 tx=0).stop 2024-04-16T15:41:31.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.901+0000 7fd5578ba640 1 -- 172.21.15.71:0/598998703 shutdown_connections 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.901+0000 7fd5578ba640 1 --2- 172.21.15.71:0/598998703 >> v2:172.21.15.122:3300/0 conn(0x7fd55006f9a0 0x7fd55006fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.901+0000 7fd5578ba640 1 --2- 172.21.15.71:0/598998703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5500adb90 0x7fd5500c2810 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.901+0000 7fd5578ba640 1 -- 172.21.15.71:0/598998703 >> 172.21.15.71:0/598998703 conn(0x7fd5500aa220 msgr2=0x7fd5500aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.901+0000 7fd5578ba640 1 -- 172.21.15.71:0/598998703 shutdown_connections 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.901+0000 7fd5578ba640 1 -- 172.21.15.71:0/598998703 wait complete. 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.902+0000 7fd5578ba640 1 Processor -- start 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.902+0000 7fd5578ba640 1 -- start start 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd5578ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd55006f9a0 0x7fd550149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd5578ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd550070310 0x7fd5500c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd5578ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5500adb90 0x7fd550153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd5578ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd550073710 con 0x7fd55006f9a0 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd5578ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5500c3410 con 0x7fd5500adb90 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd5578ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd550148f70 con 0x7fd550070310 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd55006f9a0 0x7fd550149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd55006f9a0 0x7fd550149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53738/0 (socket says 172.21.15.71:53738) 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 -- 172.21.15.71:0/311917643 learned_addr learned my addr 172.21.15.71:0/311917643 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555e57640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5500adb90 0x7fd550153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd554e55640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd550070310 0x7fd5500c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 -- 172.21.15.71:0/311917643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd550070310 msgr2=0x7fd5500c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd550070310 0x7fd5500c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 -- 172.21.15.71:0/311917643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5500adb90 msgr2=0x7fd550153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5500adb90 0x7fd550153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 -- 172.21.15.71:0/311917643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55014b390 con 0x7fd55006f9a0 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd554e55640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd550070310 0x7fd5500c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd555656640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd55006f9a0 0x7fd550149e90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd54c02b8c0 tx=0x7fd54c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.903+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c032880 con 0x7fd55006f9a0 2024-04-16T15:41:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.904+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c032a20 con 0x7fd55006f9a0 2024-04-16T15:41:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.904+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55014b670 con 0x7fd55006f9a0 2024-04-16T15:41:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.904+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c032e10 con 0x7fd55006f9a0 2024-04-16T15:41:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.904+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5500c7e30 con 0x7fd55006f9a0 2024-04-16T15:41:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.904+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd550074000 con 0x7fd55006f9a0 2024-04-16T15:41:31.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.905+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd54c0453f0 con 0x7fd55006f9a0 2024-04-16T15:41:31.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd545ffb640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd53406c920 0x7fd53406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd54c0add30 con 0x7fd55006f9a0 2024-04-16T15:41:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd554e55640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd53406c920 0x7fd53406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd545ffb640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7fd534075180 0x7fd5340775c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 --> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd54c037e70 con 0x7fd534075180 2024-04-16T15:41:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd54c0b0430 con 0x7fd55006f9a0 2024-04-16T15:41:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd555e57640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7fd534075180 0x7fd5340775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd554e55640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd53406c920 0x7fd53406ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd5500c2800 tx=0x7fd540008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.906+0000 7fd555e57640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7fd534075180 0x7fd5340775c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:31.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.907+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 <== osd.0 v2:172.21.15.71:6800/2582424538 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd54c037e70 con 0x7fd534075180 2024-04-16T15:41:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.920+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 --> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd550068be0 con 0x7fd534075180 2024-04-16T15:41:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.920+0000 7fd545ffb640 1 -- 172.21.15.71:0/311917643 <== osd.0 v2:172.21.15.71:6800/2582424538 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd550068be0 con 0x7fd534075180 2024-04-16T15:41:31.919 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T15:41:31.919 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:31.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.920+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7fd534075180 msgr2=0x7fd5340775c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:31.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.920+0000 7fd5578ba640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6800/2582424538,v1:172.21.15.71:6801/2582424538] conn(0x7fd534075180 0x7fd5340775c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:31.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd53406c920 msgr2=0x7fd53406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:31.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd5578ba640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd53406c920 0x7fd53406ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd5500c2800 tx=0x7fd540008040 comp rx=0 tx=0).stop 2024-04-16T15:41:31.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd55006f9a0 msgr2=0x7fd550149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:31.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd5578ba640 1 --2- 172.21.15.71:0/311917643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd55006f9a0 0x7fd550149e90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd54c02b8c0 tx=0x7fd54c0046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:31.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd555e57640 1 -- 172.21.15.71:0/311917643 reap_dead start 2024-04-16T15:41:31.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 shutdown_connections 2024-04-16T15:41:31.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 >> 172.21.15.71:0/311917643 conn(0x7fd5500aa220 msgr2=0x7fd55015e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:31.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 shutdown_connections 2024-04-16T15:41:31.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:31.921+0000 7fd5578ba640 1 -- 172.21.15.71:0/311917643 wait complete. 2024-04-16T15:41:31.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-16T15:41:32.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.038+0000 7f30bdfa8640 1 Processor -- start 2024-04-16T15:41:32.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.038+0000 7f30bdfa8640 1 -- start start 2024-04-16T15:41:32.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.038+0000 7f30bdfa8640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f30b80caf00 0x7f30b80ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.038+0000 7f30bdfa8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f30b80ca910 0x7f30b80c0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.038+0000 7f30bdfa8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30b8155c00 con 0x7f30b8153300 2024-04-16T15:41:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.038+0000 7f30bdfa8640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f30b81567a0 con 0x7f30b80caf00 2024-04-16T15:41:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.038+0000 7f30bdfa8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f30b8155e70 con 0x7f30b80ca910 2024-04-16T15:41:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b6ffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f30b80caf00 0x7f30b80ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b77fe640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f30b8153300 0x7f30b8159af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35060/0 (socket says 172.21.15.71:35060) 2024-04-16T15:41:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b77fe640 1 -- 172.21.15.71:0/1399959345 learned_addr learned my addr 172.21.15.71:0/1399959345 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b6ffd640 1 -- 172.21.15.71:0/1399959345 >> v2:172.21.15.122:3300/0 conn(0x7f30b80ca910 msgr2=0x7f30b80c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:32.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b7fff640 1 --2- 172.21.15.71:0/1399959345 >> v2:172.21.15.122:3300/0 conn(0x7f30b80ca910 0x7f30b80c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b6ffd640 1 --2- 172.21.15.71:0/1399959345 >> v2:172.21.15.122:3300/0 conn(0x7f30b80ca910 0x7f30b80c0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b6ffd640 1 -- 172.21.15.71:0/1399959345 >> v1:172.21.15.71:6789/0 conn(0x7f30b8153300 legacy=0x7f30b8159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b67fc640 1 -- 172.21.15.71:0/1399959345 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256332078 0 0) 0x7f30b8155c00 con 0x7f30b8153300 2024-04-16T15:41:32.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.039+0000 7f30b6ffd640 1 -- 172.21.15.71:0/1399959345 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b8156150 con 0x7f30b80caf00 2024-04-16T15:41:32.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30b6ffd640 1 --2- 172.21.15.71:0/1399959345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80caf00 0x7f30b80ca3d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f30b808c4b0 tx=0x7f30a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28a9372c585e0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30b67fc640 1 -- 172.21.15.71:0/1399959345 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a00315a0 con 0x7f30b80caf00 2024-04-16T15:41:32.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30b67fc640 1 -- 172.21.15.71:0/1399959345 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a0037070 con 0x7f30b80caf00 2024-04-16T15:41:32.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30b67fc640 1 -- 172.21.15.71:0/1399959345 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a0031960 con 0x7f30b80caf00 2024-04-16T15:41:32.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30bdfa8640 1 -- 172.21.15.71:0/1399959345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80caf00 msgr2=0x7f30b80ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/1399959345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80caf00 0x7f30b80ca3d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f30b808c4b0 tx=0x7f30a002b840 comp rx=0 tx=0).stop 2024-04-16T15:41:32.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30bdfa8640 1 -- 172.21.15.71:0/1399959345 shutdown_connections 2024-04-16T15:41:32.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/1399959345 >> v2:172.21.15.122:3300/0 conn(0x7f30b80ca910 0x7f30b80c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/1399959345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80caf00 0x7f30b80ca3d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30bdfa8640 1 -- 172.21.15.71:0/1399959345 >> 172.21.15.71:0/1399959345 conn(0x7f30b80c7f40 msgr2=0x7f30b80c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30bdfa8640 1 -- 172.21.15.71:0/1399959345 shutdown_connections 2024-04-16T15:41:32.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.040+0000 7f30bdfa8640 1 -- 172.21.15.71:0/1399959345 wait complete. 2024-04-16T15:41:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.041+0000 7f30bdfa8640 1 Processor -- start 2024-04-16T15:41:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30bdfa8640 1 -- start start 2024-04-16T15:41:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30bdfa8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80ca910 0x7f30b814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30bdfa8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b80caf00 0x7f30b81eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30bdfa8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30b8153300 0x7f30b81ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30bdfa8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f30b80c0a20 con 0x7f30b80ca910 2024-04-16T15:41:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30bdfa8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30b80c08a0 con 0x7f30b80caf00 2024-04-16T15:41:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30bdfa8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f30b80c0ba0 con 0x7f30b8153300 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80ca910 0x7f30b814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80ca910 0x7f30b814c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53758/0 (socket says 172.21.15.71:53758) 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.042+0000 7f30b77fe640 1 -- 172.21.15.71:0/3445836913 learned_addr learned my addr 172.21.15.71:0/3445836913 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30b77fe640 1 -- 172.21.15.71:0/3445836913 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30b8153300 msgr2=0x7f30b81ebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30b77fe640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30b8153300 0x7f30b81ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30b77fe640 1 -- 172.21.15.71:0/3445836913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b80caf00 msgr2=0x7f30b81eb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30b77fe640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b80caf00 0x7f30b81eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30b77fe640 1 -- 172.21.15.71:0/3445836913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b814adb0 con 0x7f30b80ca910 2024-04-16T15:41:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30b77fe640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80ca910 0x7f30b814c580 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c430 tx=0x7f30a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30affff640 1 -- 172.21.15.71:0/3445836913 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80074e0 con 0x7f30b80ca910 2024-04-16T15:41:32.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30affff640 1 -- 172.21.15.71:0/3445836913 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a80047e0 con 0x7f30b80ca910 2024-04-16T15:41:32.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30affff640 1 -- 172.21.15.71:0/3445836913 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8004bd0 con 0x7f30b80ca910 2024-04-16T15:41:32.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b814b090 con 0x7f30b80ca910 2024-04-16T15:41:32.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.043+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b814cac0 con 0x7f30b80ca910 2024-04-16T15:41:32.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.044+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b8068c10 con 0x7f30b80ca910 2024-04-16T15:41:32.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.045+0000 7f30affff640 1 -- 172.21.15.71:0/3445836913 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30a8011070 con 0x7f30b80ca910 2024-04-16T15:41:32.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.045+0000 7f30affff640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f309806c8f0 0x7f309806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.045+0000 7f30affff640 1 -- 172.21.15.71:0/3445836913 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f30a808b930 con 0x7f30b80ca910 2024-04-16T15:41:32.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.045+0000 7f30b6ffd640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f309806c8f0 0x7f309806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.046+0000 7f30b6ffd640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f309806c8f0 0x7f309806edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f30a0036000 tx=0x7f30a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.053+0000 7f30affff640 1 -- 172.21.15.71:0/3445836913 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30a8059c60 con 0x7f30b80ca910 2024-04-16T15:41:32.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.240+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f30b80c6e50 con 0x7f309806c8f0 2024-04-16T15:41:32.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.240+0000 7f30affff640 1 -- 172.21.15.71:0/3445836913 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f30b80c6e50 con 0x7f309806c8f0 2024-04-16T15:41:32.239 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 0 to deep-scrub 2024-04-16T15:41:32.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f309806c8f0 msgr2=0x7f309806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f309806c8f0 0x7f309806edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f30a0036000 tx=0x7f30a0030000 comp rx=0 tx=0).stop 2024-04-16T15:41:32.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80ca910 msgr2=0x7f30b814c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80ca910 0x7f30b814c580 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c430 tx=0x7f30a800c8f0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 shutdown_connections 2024-04-16T15:41:32.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f309806c8f0 0x7f309806edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f30b8153300 0x7f30b81ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b80caf00 0x7f30b81eb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 --2- 172.21.15.71:0/3445836913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b80ca910 0x7f30b814c580 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 >> 172.21.15.71:0/3445836913 conn(0x7f30b80c7f40 msgr2=0x7f30b80ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 shutdown_connections 2024-04-16T15:41:32.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.243+0000 7f30bdfa8640 1 -- 172.21.15.71:0/3445836913 wait complete. 2024-04-16T15:41:32.253 INFO:tasks.ceph:Scrubbing osd.1 2024-04-16T15:41:32.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-16T15:41:32.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.363+0000 7fab5e655640 1 Processor -- start 2024-04-16T15:41:32.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.363+0000 7fab5e655640 1 -- start start 2024-04-16T15:41:32.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.364+0000 7fab5e655640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fab58153370 0x7fab580c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.364+0000 7fab5e655640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab581594b0 con 0x7fab58155e90 2024-04-16T15:41:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.364+0000 7fab5e655640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab58159790 con 0x7fab580c9a40 2024-04-16T15:41:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.364+0000 7fab5e655640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fab58159a70 con 0x7fab58153370 2024-04-16T15:41:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.364+0000 7fab5cbf2640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fab58155e90 0x7fab581562e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35076/0 (socket says 172.21.15.71:35076) 2024-04-16T15:41:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.364+0000 7fab5cbf2640 1 -- 172.21.15.71:0/3341646671 learned_addr learned my addr 172.21.15.71:0/3341646671 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.364+0000 7fab57fff640 1 --2- 172.21.15.71:0/3341646671 >> v2:172.21.15.122:3300/0 conn(0x7fab58153370 0x7fab580c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.364+0000 7fab577fe640 1 -- 172.21.15.71:0/3341646671 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962431436 0 0) 0x7fab581594b0 con 0x7fab58155e90 2024-04-16T15:41:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.365+0000 7fab577fe640 1 -- 172.21.15.71:0/3341646671 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab44003520 con 0x7fab58155e90 2024-04-16T15:41:32.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.365+0000 7fab57fff640 1 -- 172.21.15.71:0/3341646671 >> v1:172.21.15.122:6789/0 conn(0x7fab580c9a40 legacy=0x7fab580cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.365+0000 7fab57fff640 1 -- 172.21.15.71:0/3341646671 >> v1:172.21.15.71:6789/0 conn(0x7fab58155e90 legacy=0x7fab581562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.365+0000 7fab57fff640 1 -- 172.21.15.71:0/3341646671 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab58159d50 con 0x7fab58153370 2024-04-16T15:41:32.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab57fff640 1 --2- 172.21.15.71:0/3341646671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58153370 0x7fab580c9500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab480097a0 tx=0x7fab4802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c3aa7734ccd33c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab577fe640 1 -- 172.21.15.71:0/3341646671 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab480315a0 con 0x7fab58153370 2024-04-16T15:41:32.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab577fe640 1 -- 172.21.15.71:0/3341646671 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab48037070 con 0x7fab58153370 2024-04-16T15:41:32.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab577fe640 1 -- 172.21.15.71:0/3341646671 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48031940 con 0x7fab58153370 2024-04-16T15:41:32.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab5e655640 1 -- 172.21.15.71:0/3341646671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58153370 msgr2=0x7fab580c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab5e655640 1 --2- 172.21.15.71:0/3341646671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58153370 0x7fab580c9500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab480097a0 tx=0x7fab4802b840 comp rx=0 tx=0).stop 2024-04-16T15:41:32.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab5e655640 1 -- 172.21.15.71:0/3341646671 shutdown_connections 2024-04-16T15:41:32.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab5e655640 1 --2- 172.21.15.71:0/3341646671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58153370 0x7fab580c9500 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.366+0000 7fab5e655640 1 -- 172.21.15.71:0/3341646671 >> 172.21.15.71:0/3341646671 conn(0x7fab5806bd50 msgr2=0x7fab5806c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.367+0000 7fab5e655640 1 -- 172.21.15.71:0/3341646671 shutdown_connections 2024-04-16T15:41:32.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.367+0000 7fab5e655640 1 -- 172.21.15.71:0/3341646671 wait complete. 2024-04-16T15:41:32.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab5e655640 1 Processor -- start 2024-04-16T15:41:32.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab5e655640 1 -- start start 2024-04-16T15:41:32.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab5e655640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fab580c9a40 0x7fab580ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab5e655640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab58153370 0x7fab581eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab5e655640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58155e90 0x7fab581ee300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab5e655640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab580cc260 con 0x7fab58153370 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab5e655640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab58153750 con 0x7fab58155e90 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab5e655640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fab580cc3e0 con 0x7fab580c9a40 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab58153370 0x7fab581eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.368+0000 7fab4ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab58153370 0x7fab581eddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53774/0 (socket says 172.21.15.71:53774) 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab4ffff640 1 -- 172.21.15.71:0/2654502885 learned_addr learned my addr 172.21.15.71:0/2654502885 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab57fff640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fab580c9a40 0x7fab580ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab5cbf2640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58155e90 0x7fab581ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab4ffff640 1 -- 172.21.15.71:0/2654502885 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fab580c9a40 msgr2=0x7fab580ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab4ffff640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fab580c9a40 0x7fab580ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab4ffff640 1 -- 172.21.15.71:0/2654502885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58155e90 msgr2=0x7fab581ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab4ffff640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58155e90 0x7fab581ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab4ffff640 1 -- 172.21.15.71:0/2654502885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab58157fd0 con 0x7fab58153370 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab5cbf2640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab58155e90 0x7fab581ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab4ffff640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab58153370 0x7fab581eddc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fab4000c470 tx=0x7fab40001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400052d0 con 0x7fab58153370 2024-04-16T15:41:32.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab40005470 con 0x7fab58153370 2024-04-16T15:41:32.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40007260 con 0x7fab58153370 2024-04-16T15:41:32.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab581ed000 con 0x7fab58153370 2024-04-16T15:41:32.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.369+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581ed5c0 con 0x7fab58153370 2024-04-16T15:41:32.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.370+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab580ca690 con 0x7fab58153370 2024-04-16T15:41:32.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.371+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab40007400 con 0x7fab58153370 2024-04-16T15:41:32.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab557fa640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fab3806c920 0x7fab3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fab4008b240 con 0x7fab58153370 2024-04-16T15:41:32.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab57fff640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fab3806c920 0x7fab3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab557fa640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fab38075190 0x7fab380775d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 --> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab40009e50 con 0x7fab38075190 2024-04-16T15:41:32.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fab40092050 con 0x7fab58153370 2024-04-16T15:41:32.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab5cbf2640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fab38075190 0x7fab380775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab57fff640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fab3806c920 0x7fab3806ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fab48036000 tx=0x7fab48008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.372+0000 7fab5cbf2640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fab38075190 0x7fab380775d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.373+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 <== osd.1 v2:172.21.15.71:6808/357157073 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab40009e50 con 0x7fab38075190 2024-04-16T15:41:32.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.386+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 --> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fab58068be0 con 0x7fab38075190 2024-04-16T15:41:32.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.386+0000 7fab557fa640 1 -- 172.21.15.71:0/2654502885 <== osd.1 v2:172.21.15.71:6808/357157073 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fab58068be0 con 0x7fab38075190 2024-04-16T15:41:32.385 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T15:41:32.385 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:32.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.386+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fab38075190 msgr2=0x7fab380775d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.386+0000 7fab5e655640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6808/357157073,v1:172.21.15.71:6809/357157073] conn(0x7fab38075190 0x7fab380775d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fab3806c920 msgr2=0x7fab3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5e655640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fab3806c920 0x7fab3806ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fab48036000 tx=0x7fab48008030 comp rx=0 tx=0).stop 2024-04-16T15:41:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab58153370 msgr2=0x7fab581eddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5e655640 1 --2- 172.21.15.71:0/2654502885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab58153370 0x7fab581eddc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fab4000c470 tx=0x7fab40001e70 comp rx=0 tx=0).stop 2024-04-16T15:41:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5cbf2640 1 -- 172.21.15.71:0/2654502885 reap_dead start 2024-04-16T15:41:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 shutdown_connections 2024-04-16T15:41:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 >> 172.21.15.71:0/2654502885 conn(0x7fab5806bd50 msgr2=0x7fab5806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 shutdown_connections 2024-04-16T15:41:32.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.387+0000 7fab5e655640 1 -- 172.21.15.71:0/2654502885 wait complete. 2024-04-16T15:41:32.396 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-16T15:41:32.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.503+0000 7fce341d4640 1 Processor -- start 2024-04-16T15:41:32.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.503+0000 7fce341d4640 1 -- start start 2024-04-16T15:41:32.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce341d4640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fce2c150150 0x7fce2c072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce341d4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fce2c0739e0 0x7fce2c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce341d4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2c0c3610 con 0x7fce2c0a9a90 2024-04-16T15:41:32.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce341d4640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7fce2c0c2e00 con 0x7fce2c150150 2024-04-16T15:41:32.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce341d4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fce2c0c3050 con 0x7fce2c0739e0 2024-04-16T15:41:32.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce31f70640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fce2c150150 0x7fce2c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce31f70640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7fce2c150150 0x7fce2c072960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53778/0 (socket says 172.21.15.71:53778) 2024-04-16T15:41:32.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce31f70640 1 -- 172.21.15.71:0/1165132639 learned_addr learned my addr 172.21.15.71:0/1165132639 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:32.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce31f70640 1 -- 172.21.15.71:0/1165132639 >> v2:172.21.15.122:3300/0 conn(0x7fce2c0739e0 msgr2=0x7fce2c15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:32.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce32771640 1 --2- 172.21.15.71:0/1165132639 >> v2:172.21.15.122:3300/0 conn(0x7fce2c0739e0 0x7fce2c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce31f70640 1 --2- 172.21.15.71:0/1165132639 >> v2:172.21.15.122:3300/0 conn(0x7fce2c0739e0 0x7fce2c15d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce31f70640 1 -- 172.21.15.71:0/1165132639 >> v1:172.21.15.122:6789/0 conn(0x7fce2c0a9a90 legacy=0x7fce2c153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:32.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.504+0000 7fce31f70640 1 -- 172.21.15.71:0/1165132639 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c0c3330 con 0x7fce2c150150 2024-04-16T15:41:32.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.505+0000 7fce31f70640 1 --2- 172.21.15.71:0/1165132639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c150150 0x7fce2c072960 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06dce0 tx=0x7fce2402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccddbf62b10c1c9a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.505+0000 7fce30f6e640 1 -- 172.21.15.71:0/1165132639 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2403b070 con 0x7fce2c150150 2024-04-16T15:41:32.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.505+0000 7fce30f6e640 1 -- 172.21.15.71:0/1165132639 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2402ee00 con 0x7fce2c150150 2024-04-16T15:41:32.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.505+0000 7fce30f6e640 1 -- 172.21.15.71:0/1165132639 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24036660 con 0x7fce2c150150 2024-04-16T15:41:32.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.506+0000 7fce341d4640 1 -- 172.21.15.71:0/1165132639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c150150 msgr2=0x7fce2c072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.506+0000 7fce341d4640 1 --2- 172.21.15.71:0/1165132639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c150150 0x7fce2c072960 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06dce0 tx=0x7fce2402e050 comp rx=0 tx=0).stop 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.506+0000 7fce341d4640 1 -- 172.21.15.71:0/1165132639 shutdown_connections 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.506+0000 7fce341d4640 1 --2- 172.21.15.71:0/1165132639 >> v2:172.21.15.122:3300/0 conn(0x7fce2c0739e0 0x7fce2c15d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.506+0000 7fce341d4640 1 --2- 172.21.15.71:0/1165132639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c150150 0x7fce2c072960 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.506+0000 7fce341d4640 1 -- 172.21.15.71:0/1165132639 >> 172.21.15.71:0/1165132639 conn(0x7fce2c06e6e0 msgr2=0x7fce2c06eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.506+0000 7fce341d4640 1 -- 172.21.15.71:0/1165132639 shutdown_connections 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.506+0000 7fce341d4640 1 -- 172.21.15.71:0/1165132639 wait complete. 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.507+0000 7fce341d4640 1 Processor -- start 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.507+0000 7fce341d4640 1 -- start start 2024-04-16T15:41:32.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce341d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce2c0739e0 0x7fce2c0aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce341d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c0a9a90 0x7fce2c0aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce341d4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce2c150150 0x7fce2c1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce341d4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c0c4c60 con 0x7fce2c0a9a90 2024-04-16T15:41:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce341d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c0c4ae0 con 0x7fce2c0739e0 2024-04-16T15:41:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce341d4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fce2c1537a0 con 0x7fce2c150150 2024-04-16T15:41:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c0a9a90 0x7fce2c0aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c0a9a90 0x7fce2c0aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53780/0 (socket says 172.21.15.71:53780) 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 -- 172.21.15.71:0/800595152 learned_addr learned my addr 172.21.15.71:0/800595152 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce31f70640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce2c0739e0 0x7fce2c0aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce32771640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce2c150150 0x7fce2c1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 -- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce2c150150 msgr2=0x7fce2c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce2c150150 0x7fce2c1437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 -- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce2c0739e0 msgr2=0x7fce2c0aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce2c0739e0 0x7fce2c0aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 -- 172.21.15.71:0/800595152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c1e7f10 con 0x7fce2c0a9a90 2024-04-16T15:41:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce31f70640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce2c0739e0 0x7fce2c0aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.508+0000 7fce3176f640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c0a9a90 0x7fce2c0aa690 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c7b0 tx=0x7fce2000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.509+0000 7fce2a7fc640 1 -- 172.21.15.71:0/800595152 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20016020 con 0x7fce2c0a9a90 2024-04-16T15:41:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.509+0000 7fce2a7fc640 1 -- 172.21.15.71:0/800595152 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2000aa50 con 0x7fce2c0a9a90 2024-04-16T15:41:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.509+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c1e81f0 con 0x7fce2c0a9a90 2024-04-16T15:41:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.509+0000 7fce2a7fc640 1 -- 172.21.15.71:0/800595152 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2000ae40 con 0x7fce2c0a9a90 2024-04-16T15:41:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.509+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c0c05e0 con 0x7fce2c0a9a90 2024-04-16T15:41:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.510+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c068be0 con 0x7fce2c0a9a90 2024-04-16T15:41:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.510+0000 7fce2a7fc640 1 -- 172.21.15.71:0/800595152 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce20019020 con 0x7fce2c0a9a90 2024-04-16T15:41:32.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.511+0000 7fce2a7fc640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1806c920 0x7fce1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.511+0000 7fce31f70640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1806c920 0x7fce1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.511+0000 7fce2a7fc640 1 -- 172.21.15.71:0/800595152 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fce2008bd20 con 0x7fce2c0a9a90 2024-04-16T15:41:32.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.511+0000 7fce31f70640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1806c920 0x7fce1806ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fce24038000 tx=0x7fce24032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.515+0000 7fce2a7fc640 1 -- 172.21.15.71:0/800595152 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce20020030 con 0x7fce2c0a9a90 2024-04-16T15:41:32.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.699+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fce2c0737c0 con 0x7fce1806c920 2024-04-16T15:41:32.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.699+0000 7fce2a7fc640 1 -- 172.21.15.71:0/800595152 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fce2c0737c0 con 0x7fce1806c920 2024-04-16T15:41:32.698 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 1 to deep-scrub 2024-04-16T15:41:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1806c920 msgr2=0x7fce1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1806c920 0x7fce1806ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fce24038000 tx=0x7fce24032000 comp rx=0 tx=0).stop 2024-04-16T15:41:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c0a9a90 msgr2=0x7fce2c0aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c0a9a90 0x7fce2c0aa690 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c7b0 tx=0x7fce2000cc70 comp rx=0 tx=0).stop 2024-04-16T15:41:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 shutdown_connections 2024-04-16T15:41:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fce1806c920 0x7fce1806ede0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fce2c150150 0x7fce2c1437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce2c0a9a90 0x7fce2c0aa690 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 --2- 172.21.15.71:0/800595152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fce2c0739e0 0x7fce2c0aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 >> 172.21.15.71:0/800595152 conn(0x7fce2c06e6e0 msgr2=0x7fce2c0be2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 shutdown_connections 2024-04-16T15:41:32.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.702+0000 7fce341d4640 1 -- 172.21.15.71:0/800595152 wait complete. 2024-04-16T15:41:32.711 INFO:tasks.ceph:Scrubbing osd.2 2024-04-16T15:41:32.712 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-16T15:41:32.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.822+0000 7fd9f6ab3640 1 Processor -- start 2024-04-16T15:41:32.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.822+0000 7fd9f6ab3640 1 -- start start 2024-04-16T15:41:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.822+0000 7fd9f6ab3640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd9f014c910 0x7fd9f00c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.822+0000 7fd9f6ab3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f0155fe0 0x7fd9f01563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.822+0000 7fd9f6ab3640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f0156dd0 con 0x7fd9f006bb50 2024-04-16T15:41:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.822+0000 7fd9f6ab3640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f0072410 con 0x7fd9f014c910 2024-04-16T15:41:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.822+0000 7fd9f6ab3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f00c3160 con 0x7fd9f0155fe0 2024-04-16T15:41:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.823+0000 7fd9f5050640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f0155fe0 0x7fd9f01563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.823+0000 7fd9f484f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd9f014c910 0x7fd9f00c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.823+0000 7fd9f5050640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f0155fe0 0x7fd9f01563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:53988/0 (socket says 172.21.15.71:53988) 2024-04-16T15:41:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.823+0000 7fd9f5050640 1 -- 172.21.15.71:0/1178500869 learned_addr learned my addr 172.21.15.71:0/1178500869 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:32.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.823+0000 7fd9f484f640 1 -- 172.21.15.71:0/1178500869 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f0155fe0 msgr2=0x7fd9f01563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:32.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.823+0000 7fd9f484f640 1 --2- 172.21.15.71:0/1178500869 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f0155fe0 0x7fd9f01563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.823+0000 7fd9f484f640 1 -- 172.21.15.71:0/1178500869 >> v1:172.21.15.122:6789/0 conn(0x7fd9f006bb50 legacy=0x7fd9f0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.823+0000 7fd9f484f640 1 -- 172.21.15.71:0/1178500869 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f00c2830 con 0x7fd9f014c910 2024-04-16T15:41:32.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.824+0000 7fd9f484f640 1 --2- 172.21.15.71:0/1178500869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f014c910 0x7fd9f00c35f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097a0 tx=0x7fd9e402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4d15186a14024a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.824+0000 7fd9ef7fe640 1 -- 172.21.15.71:0/1178500869 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4035020 con 0x7fd9f014c910 2024-04-16T15:41:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.824+0000 7fd9ef7fe640 1 -- 172.21.15.71:0/1178500869 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e4031830 con 0x7fd9f014c910 2024-04-16T15:41:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.824+0000 7fd9ef7fe640 1 -- 172.21.15.71:0/1178500869 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4031c00 con 0x7fd9f014c910 2024-04-16T15:41:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.825+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/1178500869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f014c910 msgr2=0x7fd9f00c35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.825+0000 7fd9f6ab3640 1 --2- 172.21.15.71:0/1178500869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f014c910 0x7fd9f00c35f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097a0 tx=0x7fd9e402b8d0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.825+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/1178500869 shutdown_connections 2024-04-16T15:41:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.825+0000 7fd9f6ab3640 1 --2- 172.21.15.71:0/1178500869 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f0155fe0 0x7fd9f01563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.825+0000 7fd9f6ab3640 1 --2- 172.21.15.71:0/1178500869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f014c910 0x7fd9f00c35f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.825+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/1178500869 >> 172.21.15.71:0/1178500869 conn(0x7fd9f00c3e80 msgr2=0x7fd9f00c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.825+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/1178500869 shutdown_connections 2024-04-16T15:41:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.825+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/1178500869 wait complete. 2024-04-16T15:41:32.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.826+0000 7fd9f6ab3640 1 Processor -- start 2024-04-16T15:41:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.826+0000 7fd9f6ab3640 1 -- start start 2024-04-16T15:41:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f6ab3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f006bb50 0x7fd9f0141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f6ab3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f014c910 0x7fd9f00c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f6ab3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f0155fe0 0x7fd9f00c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f6ab3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f0071930 con 0x7fd9f0155fe0 2024-04-16T15:41:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f6ab3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f00c2df0 con 0x7fd9f006bb50 2024-04-16T15:41:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f6ab3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f0071ab0 con 0x7fd9f014c910 2024-04-16T15:41:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f0155fe0 0x7fd9f00c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f0155fe0 0x7fd9f00c0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53792/0 (socket says 172.21.15.71:53792) 2024-04-16T15:41:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 -- 172.21.15.71:0/2731209273 learned_addr learned my addr 172.21.15.71:0/2731209273 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f484f640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f006bb50 0x7fd9f0141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9effff640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f014c910 0x7fd9f00c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 -- 172.21.15.71:0/2731209273 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f014c910 msgr2=0x7fd9f00c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9f014c910 0x7fd9f00c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 -- 172.21.15.71:0/2731209273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f006bb50 msgr2=0x7fd9f0141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f006bb50 0x7fd9f0141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 -- 172.21.15.71:0/2731209273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f006c7a0 con 0x7fd9f0155fe0 2024-04-16T15:41:32.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9f5050640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f0155fe0 0x7fd9f00c0a20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800efc0 tx=0x7fd9e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800cd20 con 0x7fd9f0155fe0 2024-04-16T15:41:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.827+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e8003990 con 0x7fd9f0155fe0 2024-04-16T15:41:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.828+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8003d80 con 0x7fd9f0155fe0 2024-04-16T15:41:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.828+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f006ca80 con 0x7fd9f0155fe0 2024-04-16T15:41:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.828+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f006d040 con 0x7fd9f0155fe0 2024-04-16T15:41:32.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.828+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd9f0068be0 con 0x7fd9f0155fe0 2024-04-16T15:41:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.829+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9e8005d10 con 0x7fd9f0155fe0 2024-04-16T15:41:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.829+0000 7fd9ed7fa640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd9d406c8f0 0x7fd9d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.830+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd9e808cd40 con 0x7fd9f0155fe0 2024-04-16T15:41:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.830+0000 7fd9f484f640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd9d406c8f0 0x7fd9d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.830+0000 7fd9ed7fa640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fd9d4075150 0x7fd9d4077590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.830+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 --> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9e801ee50 con 0x7fd9d4075150 2024-04-16T15:41:32.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.830+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd9e808d1d0 con 0x7fd9f0155fe0 2024-04-16T15:41:32.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.830+0000 7fd9effff640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fd9d4075150 0x7fd9d4077590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.830+0000 7fd9f484f640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd9d406c8f0 0x7fd9d406edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4036000 tx=0x7fd9e4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.830+0000 7fd9effff640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fd9d4075150 0x7fd9d4077590 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.831+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 <== osd.2 v2:172.21.15.71:6813/3827429750 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9e801ee50 con 0x7fd9d4075150 2024-04-16T15:41:32.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 --> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd9f006b930 con 0x7fd9d4075150 2024-04-16T15:41:32.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9ed7fa640 1 -- 172.21.15.71:0/2731209273 <== osd.2 v2:172.21.15.71:6813/3827429750 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd9f006b930 con 0x7fd9d4075150 2024-04-16T15:41:32.843 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T15:41:32.844 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:32.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fd9d4075150 msgr2=0x7fd9d4077590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9f6ab3640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6813/3827429750,v1:172.21.15.71:6815/3827429750] conn(0x7fd9d4075150 0x7fd9d4077590 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd9d406c8f0 msgr2=0x7fd9d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9f6ab3640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7fd9d406c8f0 0x7fd9d406edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4036000 tx=0x7fd9e4008030 comp rx=0 tx=0).stop 2024-04-16T15:41:32.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f0155fe0 msgr2=0x7fd9f00c0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9f6ab3640 1 --2- 172.21.15.71:0/2731209273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9f0155fe0 0x7fd9f00c0a20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800efc0 tx=0x7fd9e800c490 comp rx=0 tx=0).stop 2024-04-16T15:41:32.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.844+0000 7fd9f5050640 1 -- 172.21.15.71:0/2731209273 reap_dead start 2024-04-16T15:41:32.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.845+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 shutdown_connections 2024-04-16T15:41:32.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.845+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 >> 172.21.15.71:0/2731209273 conn(0x7fd9f00c3e80 msgr2=0x7fd9f014a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.845+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 shutdown_connections 2024-04-16T15:41:32.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.845+0000 7fd9f6ab3640 1 -- 172.21.15.71:0/2731209273 wait complete. 2024-04-16T15:41:32.854 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-16T15:41:32.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.961+0000 7f3718af8640 1 Processor -- start 2024-04-16T15:41:32.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.961+0000 7f3718af8640 1 -- start start 2024-04-16T15:41:32.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.962+0000 7f3718af8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f37140caf00 0x7f37140ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.962+0000 7f3718af8640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3714155c00 con 0x7f37140ca910 2024-04-16T15:41:32.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.962+0000 7f3718af8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3714155e70 con 0x7f3714153300 2024-04-16T15:41:32.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.962+0000 7f3718af8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f37141567a0 con 0x7f37140caf00 2024-04-16T15:41:32.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.962+0000 7f3712d9d640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f37140ca910 0x7f37140c0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35088/0 (socket says 172.21.15.71:35088) 2024-04-16T15:41:32.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.962+0000 7f3712d9d640 1 -- 172.21.15.71:0/1148369666 learned_addr learned my addr 172.21.15.71:0/1148369666 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f370bfff640 1 --2- 172.21.15.71:0/1148369666 >> v2:172.21.15.122:3300/0 conn(0x7f37140caf00 0x7f37140ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173050885 0 0) 0x7f3714155c00 con 0x7f37140ca910 2024-04-16T15:41:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3700003520 con 0x7f37140ca910 2024-04-16T15:41:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2937977828 0 0) 0x7f3700003520 con 0x7f37140ca910 2024-04-16T15:41:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 >> v2:172.21.15.122:3300/0 conn(0x7f37140caf00 msgr2=0x7f37140ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 --2- 172.21.15.71:0/1148369666 >> v2:172.21.15.122:3300/0 conn(0x7f37140caf00 0x7f37140ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 >> v1:172.21.15.122:6789/0 conn(0x7f3714153300 legacy=0x7f3714159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3714156150 con 0x7f37140ca910 2024-04-16T15:41:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f370c0019f0 con 0x7f37140ca910 2024-04-16T15:41:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 >> v1:172.21.15.71:6789/0 conn(0x7f37140ca910 legacy=0x7f37140c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000039b0 0x7f3700003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3700004800 0x7f3700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37000052e0 0x7f370000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37141567a0 con 0x7f37000052e0 2024-04-16T15:41:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f370000c1b0 con 0x7f37000039b0 2024-04-16T15:41:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f370000c3b0 con 0x7f3700004800 2024-04-16T15:41:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3712d9d640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37000052e0 0x7f370000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f371259c640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000039b0 0x7f3700003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f370bfff640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3700004800 0x7f3700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3712d9d640 1 -- 172.21.15.71:0/1148369666 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3700004800 msgr2=0x7f3700004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3712d9d640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3700004800 0x7f3700004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3712d9d640 1 -- 172.21.15.71:0/1148369666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000039b0 msgr2=0x7f3700003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3712d9d640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000039b0 0x7f3700003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.963+0000 7f3712d9d640 1 -- 172.21.15.71:0/1148369666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3700003520 con 0x7f37000052e0 2024-04-16T15:41:32.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.964+0000 7f3712d9d640 1 -- 172.21.15.71:0/1148369666 reap_dead start 2024-04-16T15:41:32.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.964+0000 7f3712d9d640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37000052e0 0x7f370000b9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f370c00a830 tx=0x7f3700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ea3bb4604616df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.964+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c00a480 con 0x7f37000052e0 2024-04-16T15:41:32.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.964+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f370c034630 con 0x7f37000052e0 2024-04-16T15:41:32.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.964+0000 7f3711d9b640 1 -- 172.21.15.71:0/1148369666 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c034a20 con 0x7f37000052e0 2024-04-16T15:41:32.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.965+0000 7f3718af8640 1 -- 172.21.15.71:0/1148369666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37000052e0 msgr2=0x7f370000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.965+0000 7f3718af8640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37000052e0 0x7f370000b9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f370c00a830 tx=0x7f3700005830 comp rx=0 tx=0).stop 2024-04-16T15:41:32.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.965+0000 7f3718af8640 1 -- 172.21.15.71:0/1148369666 shutdown_connections 2024-04-16T15:41:32.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.965+0000 7f3718af8640 1 --2- 172.21.15.71:0/1148369666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37000052e0 0x7f370000b9c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.965+0000 7f3718af8640 1 -- 172.21.15.71:0/1148369666 >> 172.21.15.71:0/1148369666 conn(0x7f37140c7f40 msgr2=0x7f37140c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:32.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.965+0000 7f3718af8640 1 -- 172.21.15.71:0/1148369666 shutdown_connections 2024-04-16T15:41:32.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.965+0000 7f3718af8640 1 -- 172.21.15.71:0/1148369666 wait complete. 2024-04-16T15:41:32.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.966+0000 7f3718af8640 1 Processor -- start 2024-04-16T15:41:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.966+0000 7f3718af8640 1 -- start start 2024-04-16T15:41:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3718af8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000052e0 0x7f37141ec370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3718af8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f371414f4b0 0x7f37141582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3718af8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37141587f0 0x7f37141eee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3718af8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37140ca550 con 0x7f37141587f0 2024-04-16T15:41:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3718af8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37140ca3d0 con 0x7f37000052e0 2024-04-16T15:41:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3718af8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37140ca6d0 con 0x7f371414f4b0 2024-04-16T15:41:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37141587f0 0x7f37141eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37141587f0 0x7f37141eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53810/0 (socket says 172.21.15.71:53810) 2024-04-16T15:41:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 -- 172.21.15.71:0/354923548 learned_addr learned my addr 172.21.15.71:0/354923548 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f371259c640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000052e0 0x7f37141ec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f370bfff640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f371414f4b0 0x7f37141582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 -- 172.21.15.71:0/354923548 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f371414f4b0 msgr2=0x7f37141582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f371414f4b0 0x7f37141582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 -- 172.21.15.71:0/354923548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000052e0 msgr2=0x7f37141ec370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000052e0 0x7f37141ec370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 -- 172.21.15.71:0/354923548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3714150670 con 0x7f37141587f0 2024-04-16T15:41:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.967+0000 7f3712d9d640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37141587f0 0x7f37141eee60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f370c000f80 tx=0x7f3714155af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.968+0000 7f370affd640 1 -- 172.21.15.71:0/354923548 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c045070 con 0x7f37141587f0 2024-04-16T15:41:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.968+0000 7f370affd640 1 -- 172.21.15.71:0/354923548 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f370c03a040 con 0x7f37141587f0 2024-04-16T15:41:32.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.968+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371414e320 con 0x7f37141587f0 2024-04-16T15:41:32.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.968+0000 7f370affd640 1 -- 172.21.15.71:0/354923548 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c00a480 con 0x7f37141587f0 2024-04-16T15:41:32.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.968+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3714150390 con 0x7f37141587f0 2024-04-16T15:41:32.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.969+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3714068c10 con 0x7f37141587f0 2024-04-16T15:41:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.969+0000 7f370affd640 1 -- 172.21.15.71:0/354923548 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f370c048070 con 0x7f37141587f0 2024-04-16T15:41:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.970+0000 7f370affd640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f36f406c920 0x7f36f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.970+0000 7f370affd640 1 -- 172.21.15.71:0/354923548 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f370c0ad620 con 0x7f37141587f0 2024-04-16T15:41:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.970+0000 7f371259c640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f36f406c920 0x7f36f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.970+0000 7f371259c640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f36f406c920 0x7f36f406ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3704000c00 tx=0x7f3704009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:32.978+0000 7f370affd640 1 -- 172.21.15.71:0/354923548 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f371414e320 con 0x7f37141587f0 2024-04-16T15:41:33.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.156+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f371414fe60 con 0x7f36f406c920 2024-04-16T15:41:33.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.157+0000 7f370affd640 1 -- 172.21.15.71:0/354923548 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f371414fe60 con 0x7f36f406c920 2024-04-16T15:41:33.157 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 2 to deep-scrub 2024-04-16T15:41:33.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f36f406c920 msgr2=0x7f36f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f36f406c920 0x7f36f406ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3704000c00 tx=0x7f3704009040 comp rx=0 tx=0).stop 2024-04-16T15:41:33.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37141587f0 msgr2=0x7f37141eee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37141587f0 0x7f37141eee60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f370c000f80 tx=0x7f3714155af0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 shutdown_connections 2024-04-16T15:41:33.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f36f406c920 0x7f36f406ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37141587f0 0x7f37141eee60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f371414f4b0 0x7f37141582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 --2- 172.21.15.71:0/354923548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37000052e0 0x7f37141ec370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 >> 172.21.15.71:0/354923548 conn(0x7f37140c7f40 msgr2=0x7f37141df150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:33.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 shutdown_connections 2024-04-16T15:41:33.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.159+0000 7f3718af8640 1 -- 172.21.15.71:0/354923548 wait complete. 2024-04-16T15:41:33.170 INFO:tasks.ceph:Scrubbing osd.3 2024-04-16T15:41:33.171 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-16T15:41:33.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3bf60e4640 1 Processor -- start 2024-04-16T15:41:33.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3bf60e4640 1 -- start start 2024-04-16T15:41:33.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3bf60e4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 0x7f3bf00c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3bf60e4640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f3bf006bb50 0x7f3bf0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3bf60e4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3bf0155fe0 0x7f3bf0156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3bf60e4640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f3bf00c3160 con 0x7f3bf006bb50 2024-04-16T15:41:33.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3bf60e4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f3bf0156dd0 con 0x7f3bf0155fe0 2024-04-16T15:41:33.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3bf60e4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf00c2830 con 0x7f3bf014c910 2024-04-16T15:41:33.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.280+0000 7f3beeffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f3bf006bb50 0x7f3bf0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beeffd640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f3bf006bb50 0x7f3bf0071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53818/0 (socket says 172.21.15.71:53818) 2024-04-16T15:41:33.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beeffd640 1 -- 172.21.15.71:0/3994298810 learned_addr learned my addr 172.21.15.71:0/3994298810 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beffff640 1 --2- 172.21.15.71:0/3994298810 >> v2:172.21.15.122:3300/0 conn(0x7f3bf0155fe0 0x7f3bf0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3bef7fe640 1 --2- 172.21.15.71:0/3994298810 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 0x7f3bf00c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beeffd640 1 -- 172.21.15.71:0/3994298810 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 msgr2=0x7f3bf00c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beeffd640 1 --2- 172.21.15.71:0/3994298810 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 0x7f3bf00c35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beeffd640 1 -- 172.21.15.71:0/3994298810 >> v2:172.21.15.122:3300/0 conn(0x7f3bf0155fe0 msgr2=0x7f3bf0156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beeffd640 1 --2- 172.21.15.71:0/3994298810 >> v2:172.21.15.122:3300/0 conn(0x7f3bf0155fe0 0x7f3bf0156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beeffd640 1 -- 172.21.15.71:0/3994298810 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf00c2b10 con 0x7f3bf006bb50 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.281+0000 7f3beffff640 1 --2- 172.21.15.71:0/3994298810 >> v2:172.21.15.122:3300/0 conn(0x7f3bf0155fe0 0x7f3bf0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:33.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3beeffd640 1 --2- 172.21.15.71:0/3994298810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 0x7f3bf0071d00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00c4c50 tx=0x7f3bd802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e4a13e6099ea8c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bee7fc640 1 -- 172.21.15.71:0/3994298810 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8035020 con 0x7f3bf006bb50 2024-04-16T15:41:33.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bee7fc640 1 -- 172.21.15.71:0/3994298810 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd8031870 con 0x7f3bf006bb50 2024-04-16T15:41:33.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bee7fc640 1 -- 172.21.15.71:0/3994298810 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8031c40 con 0x7f3bf006bb50 2024-04-16T15:41:33.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 -- 172.21.15.71:0/3994298810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 msgr2=0x7f3bf0071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 --2- 172.21.15.71:0/3994298810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 0x7f3bf0071d00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00c4c50 tx=0x7f3bd802b910 comp rx=0 tx=0).stop 2024-04-16T15:41:33.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 -- 172.21.15.71:0/3994298810 shutdown_connections 2024-04-16T15:41:33.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 --2- 172.21.15.71:0/3994298810 >> v2:172.21.15.122:3300/0 conn(0x7f3bf0155fe0 0x7f3bf0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 --2- 172.21.15.71:0/3994298810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 0x7f3bf0071d00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 --2- 172.21.15.71:0/3994298810 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 0x7f3bf00c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 -- 172.21.15.71:0/3994298810 >> 172.21.15.71:0/3994298810 conn(0x7f3bf00c3e80 msgr2=0x7f3bf00c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:33.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 -- 172.21.15.71:0/3994298810 shutdown_connections 2024-04-16T15:41:33.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.282+0000 7f3bf60e4640 1 -- 172.21.15.71:0/3994298810 wait complete. 2024-04-16T15:41:33.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.283+0000 7f3bf60e4640 1 Processor -- start 2024-04-16T15:41:33.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bf60e4640 1 -- start start 2024-04-16T15:41:33.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bf60e4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 0x7f3bf01480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bf60e4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 0x7f3bf00abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bf60e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf0155fe0 0x7f3bf0155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bf60e4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0071ab0 con 0x7f3bf006bb50 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bf60e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0071930 con 0x7f3bf0155fe0 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bf60e4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf0072240 con 0x7f3bf014c910 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bef7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 0x7f3bf01480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bef7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 0x7f3bf01480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53826/0 (socket says 172.21.15.71:53826) 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.284+0000 7f3bef7fe640 1 -- 172.21.15.71:0/2161381829 learned_addr learned my addr 172.21.15.71:0/2161381829 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:33.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3beeffd640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 0x7f3bf00abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3beffff640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf0155fe0 0x7f3bf0155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3bef7fe640 1 -- 172.21.15.71:0/2161381829 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 msgr2=0x7f3bf00abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3bef7fe640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 0x7f3bf00abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3bef7fe640 1 -- 172.21.15.71:0/2161381829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf0155fe0 msgr2=0x7f3bf0155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3bef7fe640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf0155fe0 0x7f3bf0155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3bef7fe640 1 -- 172.21.15.71:0/2161381829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf014ce40 con 0x7f3bf006bb50 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3beeffd640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3bf014c910 0x7f3bf00abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3beffff640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bf0155fe0 0x7f3bf0155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3bef7fe640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 0x7f3bf01480a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007e60 tx=0x7f3be000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0014620 con 0x7f3bf006bb50 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be0019070 con 0x7f3bf006bb50 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0014a00 con 0x7f3bf006bb50 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf00ac890 con 0x7f3bf006bb50 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.285+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf00ace50 con 0x7f3bf006bb50 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.286+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3bf014e3b0 con 0x7f3bf006bb50 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.286+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3be0014ba0 con 0x7f3bf006bb50 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3be7fff640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3bd006c8f0 0x7f3bd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f3be008c5e0 con 0x7f3bf006bb50 2024-04-16T15:41:33.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3beeffd640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3bd006c8f0 0x7f3bd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3be7fff640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f3bd00750e0 0x7f3bd0077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 --> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3bd0077bf0 con 0x7f3bd00750e0 2024-04-16T15:41:33.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3be0093050 con 0x7f3bf006bb50 2024-04-16T15:41:33.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3beffff640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f3bd00750e0 0x7f3bd0077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3beeffd640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3bd006c8f0 0x7f3bd006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8036000 tx=0x7f3bd8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.287+0000 7f3beffff640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f3bd00750e0 0x7f3bd0077520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.288+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 <== osd.3 v2:172.21.15.71:6824/2052589418 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bd0077bf0 con 0x7f3bd00750e0 2024-04-16T15:41:33.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.301+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 --> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3bf0068be0 con 0x7f3bd00750e0 2024-04-16T15:41:33.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.301+0000 7f3be7fff640 1 -- 172.21.15.71:0/2161381829 <== osd.3 v2:172.21.15.71:6824/2052589418 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3bf0068be0 con 0x7f3bd00750e0 2024-04-16T15:41:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T15:41:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:33.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.301+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f3bd00750e0 msgr2=0x7f3bd0077520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.301+0000 7f3bf60e4640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6824/2052589418,v1:172.21.15.71:6825/2052589418] conn(0x7f3bd00750e0 0x7f3bd0077520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3bd006c8f0 msgr2=0x7f3bd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3bf60e4640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f3bd006c8f0 0x7f3bd006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8036000 tx=0x7f3bd8008030 comp rx=0 tx=0).stop 2024-04-16T15:41:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 msgr2=0x7f3bf01480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3bf60e4640 1 --2- 172.21.15.71:0/2161381829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bf006bb50 0x7f3bf01480a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007e60 tx=0x7f3be000e4b0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3beffff640 1 -- 172.21.15.71:0/2161381829 reap_dead start 2024-04-16T15:41:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 shutdown_connections 2024-04-16T15:41:33.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 >> 172.21.15.71:0/2161381829 conn(0x7f3bf00c3e80 msgr2=0x7f3bf0156430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:33.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 shutdown_connections 2024-04-16T15:41:33.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.302+0000 7f3bf60e4640 1 -- 172.21.15.71:0/2161381829 wait complete. 2024-04-16T15:41:33.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-16T15:41:33.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.416+0000 7f631ade2640 1 Processor -- start 2024-04-16T15:41:33.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.416+0000 7f631ade2640 1 -- start start 2024-04-16T15:41:33.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.417+0000 7f631ade2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f63140c0360 0x7f63140c07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.417+0000 7f631ade2640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6314155c00 con 0x7f63140caf00 2024-04-16T15:41:33.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.417+0000 7f631ade2640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6314155e70 con 0x7f6314153300 2024-04-16T15:41:33.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.417+0000 7f631ade2640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f63141567a0 con 0x7f63140c0360 2024-04-16T15:41:33.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.417+0000 7f6313fff640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f63140caf00 0x7f63140ca3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35092/0 (socket says 172.21.15.71:35092) 2024-04-16T15:41:33.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.417+0000 7f6313fff640 1 -- 172.21.15.71:0/2672232826 learned_addr learned my addr 172.21.15.71:0/2672232826 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:33.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f631937f640 1 --2- 172.21.15.71:0/2672232826 >> v2:172.21.15.122:3300/0 conn(0x7f63140c0360 0x7f63140c07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1834506868 0 0) 0x7f6314155c00 con 0x7f63140caf00 2024-04-16T15:41:33.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6300003520 con 0x7f63140caf00 2024-04-16T15:41:33.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 <== mon.0 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1175163528 0 0) 0x7f6300003520 con 0x7f63140caf00 2024-04-16T15:41:33.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 >> v2:172.21.15.122:3300/0 conn(0x7f63140c0360 msgr2=0x7f63140c07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:33.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 --2- 172.21.15.71:0/2672232826 >> v2:172.21.15.122:3300/0 conn(0x7f63140c0360 0x7f63140c07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 >> v1:172.21.15.122:6789/0 conn(0x7f6314153300 legacy=0x7f6314159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6314156150 con 0x7f63140caf00 2024-04-16T15:41:33.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 <== mon.0 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 235259017 0 0) 0x7f62fc001670 con 0x7f63140caf00 2024-04-16T15:41:33.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 >> v1:172.21.15.71:6789/0 conn(0x7f63140caf00 legacy=0x7f63140ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63000039b0 0x7f6300003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6300004800 0x7f6300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000051c0 0x7f630000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f63141567a0 con 0x7f63000039b0 2024-04-16T15:41:33.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f630000c160 con 0x7f63000051c0 2024-04-16T15:41:33.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f630000c360 con 0x7f6300004800 2024-04-16T15:41:33.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f6318b7e640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63000039b0 0x7f6300003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f631937f640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000051c0 0x7f630000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f6313fff640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6300004800 0x7f6300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f6318b7e640 1 -- 172.21.15.71:0/2672232826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6300004800 msgr2=0x7f6300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f6318b7e640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6300004800 0x7f6300004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f6318b7e640 1 -- 172.21.15.71:0/2672232826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000051c0 msgr2=0x7f630000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f6318b7e640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000051c0 0x7f630000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f631937f640 1 -- 172.21.15.71:0/2672232826 reap_dead start 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.418+0000 7f6318b7e640 1 -- 172.21.15.71:0/2672232826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6300003520 con 0x7f63000039b0 2024-04-16T15:41:33.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.419+0000 7f6318b7e640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63000039b0 0x7f6300003dd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6304001e50 tx=0x7f6300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee979f328e82f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.419+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304001a70 con 0x7f63000039b0 2024-04-16T15:41:33.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.419+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6304003b70 con 0x7f63000039b0 2024-04-16T15:41:33.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.419+0000 7f63137fe640 1 -- 172.21.15.71:0/2672232826 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040050d0 con 0x7f63000039b0 2024-04-16T15:41:33.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.420+0000 7f631ade2640 1 -- 172.21.15.71:0/2672232826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63000039b0 msgr2=0x7f6300003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.420+0000 7f631ade2640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63000039b0 0x7f6300003dd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6304001e50 tx=0x7f6300005830 comp rx=0 tx=0).stop 2024-04-16T15:41:33.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.420+0000 7f631ade2640 1 -- 172.21.15.71:0/2672232826 shutdown_connections 2024-04-16T15:41:33.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.420+0000 7f631ade2640 1 --2- 172.21.15.71:0/2672232826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63000039b0 0x7f6300003dd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.420+0000 7f631ade2640 1 -- 172.21.15.71:0/2672232826 >> 172.21.15.71:0/2672232826 conn(0x7f63140c7f40 msgr2=0x7f63140c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:33.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.420+0000 7f631ade2640 1 -- 172.21.15.71:0/2672232826 shutdown_connections 2024-04-16T15:41:33.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.420+0000 7f631ade2640 1 -- 172.21.15.71:0/2672232826 wait complete. 2024-04-16T15:41:33.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.421+0000 7f631ade2640 1 Processor -- start 2024-04-16T15:41:33.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.421+0000 7f631ade2640 1 -- start start 2024-04-16T15:41:33.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f631ade2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000039b0 0x7f63140c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f631ade2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6314157fc0 0x7f6314141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f631ade2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6314158440 0x7f6314142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f631ade2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6314159c70 con 0x7f6314157fc0 2024-04-16T15:41:33.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f631ade2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6314159af0 con 0x7f63000039b0 2024-04-16T15:41:33.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f631ade2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6314159df0 con 0x7f6314158440 2024-04-16T15:41:33.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6314157fc0 0x7f6314141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6314157fc0 0x7f6314141dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53842/0 (socket says 172.21.15.71:53842) 2024-04-16T15:41:33.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 -- 172.21.15.71:0/2614412965 learned_addr learned my addr 172.21.15.71:0/2614412965 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:33.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6318b7e640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000039b0 0x7f63140c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f631937f640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6314158440 0x7f6314142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 -- 172.21.15.71:0/2614412965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6314158440 msgr2=0x7f6314142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6314158440 0x7f6314142300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 -- 172.21.15.71:0/2614412965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000039b0 msgr2=0x7f63140c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000039b0 0x7f63140c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 -- 172.21.15.71:0/2614412965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63141ef780 con 0x7f6314157fc0 2024-04-16T15:41:33.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f6313fff640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6314157fc0 0x7f6314141dc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00e830 tx=0x7f62fc00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.422+0000 7f63117fa640 1 -- 172.21.15.71:0/2614412965 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc00ee20 con 0x7f6314157fc0 2024-04-16T15:41:33.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.423+0000 7f63117fa640 1 -- 172.21.15.71:0/2614412965 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc018070 con 0x7f6314157fc0 2024-04-16T15:41:33.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.423+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63141efa60 con 0x7f6314157fc0 2024-04-16T15:41:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.423+0000 7f63117fa640 1 -- 172.21.15.71:0/2614412965 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc00cb10 con 0x7f6314157fc0 2024-04-16T15:41:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.423+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63141f0020 con 0x7f6314157fc0 2024-04-16T15:41:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.424+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6314068c10 con 0x7f6314157fc0 2024-04-16T15:41:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.424+0000 7f63117fa640 1 -- 172.21.15.71:0/2614412965 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62fc00ccb0 con 0x7f6314157fc0 2024-04-16T15:41:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.424+0000 7f63117fa640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f62f406c920 0x7f62f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.425+0000 7f63117fa640 1 -- 172.21.15.71:0/2614412965 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f62fc08b650 con 0x7f6314157fc0 2024-04-16T15:41:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.425+0000 7f6318b7e640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f62f406c920 0x7f62f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.425+0000 7f6318b7e640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f62f406c920 0x7f62f406ede0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6304034000 tx=0x7f6304009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.429+0000 7f63117fa640 1 -- 172.21.15.71:0/2614412965 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62fc059980 con 0x7f6314157fc0 2024-04-16T15:41:33.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.607+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f6314152a60 con 0x7f62f406c920 2024-04-16T15:41:33.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.607+0000 7f63117fa640 1 -- 172.21.15.71:0/2614412965 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6314152a60 con 0x7f62f406c920 2024-04-16T15:41:33.607 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 3 to deep-scrub 2024-04-16T15:41:33.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f62f406c920 msgr2=0x7f62f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f62f406c920 0x7f62f406ede0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6304034000 tx=0x7f6304009030 comp rx=0 tx=0).stop 2024-04-16T15:41:33.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6314157fc0 msgr2=0x7f6314141dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6314157fc0 0x7f6314141dc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00e830 tx=0x7f62fc00ecf0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 shutdown_connections 2024-04-16T15:41:33.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f62f406c920 0x7f62f406ede0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6314158440 0x7f6314142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6314157fc0 0x7f6314141dc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 --2- 172.21.15.71:0/2614412965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63000039b0 0x7f63140c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 >> 172.21.15.71:0/2614412965 conn(0x7f63140c7f40 msgr2=0x7f63140c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:33.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 shutdown_connections 2024-04-16T15:41:33.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.609+0000 7f631ade2640 1 -- 172.21.15.71:0/2614412965 wait complete. 2024-04-16T15:41:33.620 INFO:tasks.ceph:Scrubbing osd.4 2024-04-16T15:41:33.621 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-16T15:41:33.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.728+0000 7f77b8814640 1 Processor -- start 2024-04-16T15:41:33.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.728+0000 7f77b8814640 1 -- start start 2024-04-16T15:41:33.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.729+0000 7f77b8814640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f77b4073670 0x7f77b4073a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.729+0000 7f77b8814640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f77b40c3880 0x7f77b4153950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.729+0000 7f77b8814640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b40c3d80 con 0x7f77b40c3080 2024-04-16T15:41:33.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.729+0000 7f77b8814640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f77b40c4060 con 0x7f77b4073670 2024-04-16T15:41:33.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.729+0000 7f77b8814640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f77b40adfb0 con 0x7f77b40c3880 2024-04-16T15:41:33.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.729+0000 7f77b259c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f77b4073670 0x7f77b4073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.729+0000 7f77b2d9d640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f77b40c3080 0x7f77b4153e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35108/0 (socket says 172.21.15.71:35108) 2024-04-16T15:41:33.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.729+0000 7f77b2d9d640 1 -- 172.21.15.71:0/502941601 learned_addr learned my addr 172.21.15.71:0/502941601 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:33.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.730+0000 7f77b259c640 1 -- 172.21.15.71:0/502941601 >> v2:172.21.15.122:3300/0 conn(0x7f77b40c3880 msgr2=0x7f77b4153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:33.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.730+0000 7f77b1d9b640 1 --2- 172.21.15.71:0/502941601 >> v2:172.21.15.122:3300/0 conn(0x7f77b40c3880 0x7f77b4153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.730+0000 7f77b259c640 1 --2- 172.21.15.71:0/502941601 >> v2:172.21.15.122:3300/0 conn(0x7f77b40c3880 0x7f77b4153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.730+0000 7f77b259c640 1 -- 172.21.15.71:0/502941601 >> v1:172.21.15.71:6789/0 conn(0x7f77b40c3080 legacy=0x7f77b4153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.730+0000 7f77b159a640 1 -- 172.21.15.71:0/502941601 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1077167930 0 0) 0x7f77b40c3d80 con 0x7f77b40c3080 2024-04-16T15:41:33.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.730+0000 7f77b259c640 1 -- 172.21.15.71:0/502941601 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b40ad960 con 0x7f77b4073670 2024-04-16T15:41:33.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.730+0000 7f77b259c640 1 --2- 172.21.15.71:0/502941601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 0x7f77b4073a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f77b4074780 tx=0x7f77a402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bb2b7f3b609d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.730+0000 7f77b159a640 1 -- 172.21.15.71:0/502941601 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4039070 con 0x7f77b4073670 2024-04-16T15:41:33.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b159a640 1 -- 172.21.15.71:0/502941601 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a402cb60 con 0x7f77b4073670 2024-04-16T15:41:33.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b159a640 1 -- 172.21.15.71:0/502941601 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4034440 con 0x7f77b4073670 2024-04-16T15:41:33.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b8814640 1 -- 172.21.15.71:0/502941601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 msgr2=0x7f77b4073a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b8814640 1 --2- 172.21.15.71:0/502941601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 0x7f77b4073a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f77b4074780 tx=0x7f77a402bdb0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b8814640 1 -- 172.21.15.71:0/502941601 shutdown_connections 2024-04-16T15:41:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b8814640 1 --2- 172.21.15.71:0/502941601 >> v2:172.21.15.122:3300/0 conn(0x7f77b40c3880 0x7f77b4153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b8814640 1 --2- 172.21.15.71:0/502941601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 0x7f77b4073a50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b8814640 1 -- 172.21.15.71:0/502941601 >> 172.21.15.71:0/502941601 conn(0x7f77b40aa220 msgr2=0x7f77b40aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b8814640 1 -- 172.21.15.71:0/502941601 shutdown_connections 2024-04-16T15:41:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.731+0000 7f77b8814640 1 -- 172.21.15.71:0/502941601 wait complete. 2024-04-16T15:41:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.732+0000 7f77b8814640 1 Processor -- start 2024-04-16T15:41:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.732+0000 7f77b8814640 1 -- start start 2024-04-16T15:41:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b8814640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 0x7f77b40c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b8814640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77b40c3080 0x7f77b4155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b8814640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77b40c3880 0x7f77b4149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b8814640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77b40c2a10 con 0x7f77b4073670 2024-04-16T15:41:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b8814640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77b41545a0 con 0x7f77b40c3880 2024-04-16T15:41:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b8814640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f77b41511b0 con 0x7f77b40c3080 2024-04-16T15:41:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 0x7f77b40c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 0x7f77b40c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53858/0 (socket says 172.21.15.71:53858) 2024-04-16T15:41:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b259c640 1 -- 172.21.15.71:0/1531769000 learned_addr learned my addr 172.21.15.71:0/1531769000 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b1d9b640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77b40c3080 0x7f77b4155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b259c640 1 -- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77b40c3080 msgr2=0x7f77b4155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b259c640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f77b40c3080 0x7f77b4155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b2d9d640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77b40c3880 0x7f77b4149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b259c640 1 -- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77b40c3880 msgr2=0x7f77b4149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b259c640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77b40c3880 0x7f77b4149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b259c640 1 -- 172.21.15.71:0/1531769000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b4154a90 con 0x7f77b4073670 2024-04-16T15:41:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.733+0000 7f77b2d9d640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77b40c3880 0x7f77b4149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.734+0000 7f77b259c640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 0x7f77b40c76d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009e80 tx=0x7f77a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.734+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4039050 con 0x7f77b4073670 2024-04-16T15:41:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.734+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a4045070 con 0x7f77b4073670 2024-04-16T15:41:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.734+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a403ea50 con 0x7f77b4073670 2024-04-16T15:41:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.734+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b40c6bf0 con 0x7f77b4073670 2024-04-16T15:41:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.734+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b40700c0 con 0x7f77b4073670 2024-04-16T15:41:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.735+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77b4068be0 con 0x7f77b4073670 2024-04-16T15:41:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.735+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77a402cd10 con 0x7f77b4073670 2024-04-16T15:41:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.736+0000 7f77aaffd640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f779406c8f0 0x7f779406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.736+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f77a40ae8d0 con 0x7f77b4073670 2024-04-16T15:41:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.736+0000 7f77b1d9b640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f779406c8f0 0x7f779406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.736+0000 7f77aaffd640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f7794075160 0x7f77940775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.736+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 --> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7794077c70 con 0x7f7794075160 2024-04-16T15:41:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.736+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f77a40b32c0 con 0x7f77b4073670 2024-04-16T15:41:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.736+0000 7f77b2d9d640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f7794075160 0x7f77940775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.736+0000 7f77b1d9b640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f779406c8f0 0x7f779406edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77b41ea630 tx=0x7f779c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.737+0000 7f77b2d9d640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f7794075160 0x7f77940775a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.737+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 <== osd.4 v2:172.21.15.122:6816/710391503 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7794077c70 con 0x7f7794075160 2024-04-16T15:41:33.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.750+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 --> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f77b40c3660 con 0x7f7794075160 2024-04-16T15:41:33.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77aaffd640 1 -- 172.21.15.71:0/1531769000 <== osd.4 v2:172.21.15.122:6816/710391503 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f77b40c3660 con 0x7f7794075160 2024-04-16T15:41:33.751 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:33.752 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T15:41:33.752 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f7794075160 msgr2=0x7f77940775a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b8814640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.122:6816/710391503,v1:172.21.15.122:6818/710391503] conn(0x7f7794075160 0x7f77940775a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f779406c8f0 msgr2=0x7f779406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b8814640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f779406c8f0 0x7f779406edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77b41ea630 tx=0x7f779c008040 comp rx=0 tx=0).stop 2024-04-16T15:41:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 msgr2=0x7f77b40c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b8814640 1 --2- 172.21.15.71:0/1531769000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77b4073670 0x7f77b40c76d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009e80 tx=0x7f77a4004a40 comp rx=0 tx=0).stop 2024-04-16T15:41:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b2d9d640 1 -- 172.21.15.71:0/1531769000 reap_dead start 2024-04-16T15:41:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 shutdown_connections 2024-04-16T15:41:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.751+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 >> 172.21.15.71:0/1531769000 conn(0x7f77b40aa220 msgr2=0x7f77b414f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.752+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 shutdown_connections 2024-04-16T15:41:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.752+0000 7f77b8814640 1 -- 172.21.15.71:0/1531769000 wait complete. 2024-04-16T15:41:33.763 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-16T15:41:33.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.866+0000 7f7eb45ba640 1 Processor -- start 2024-04-16T15:41:33.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.866+0000 7f7eb45ba640 1 -- start start 2024-04-16T15:41:33.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.867+0000 7f7eb45ba640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f7eac0c3600 0x7f7eac153a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.867+0000 7f7eb45ba640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac06bb80 con 0x7f7eac14d9f0 2024-04-16T15:41:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.867+0000 7f7eb45ba640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac06be60 con 0x7f7eac14d540 2024-04-16T15:41:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.867+0000 7f7eb45ba640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f7eac06b930 con 0x7f7eac0c3600 2024-04-16T15:41:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.868+0000 7f7eb2356640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f7eac0c3600 0x7f7eac153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.868+0000 7f7eb2b57640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f7eac14d9f0 0x7f7eac073620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35112/0 (socket says 172.21.15.71:35112) 2024-04-16T15:41:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.868+0000 7f7eb2b57640 1 -- 172.21.15.71:0/912981534 learned_addr learned my addr 172.21.15.71:0/912981534 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.868+0000 7f7eb2356640 1 -- 172.21.15.71:0/912981534 >> v1:172.21.15.122:6789/0 conn(0x7f7eac14d540 legacy=0x7f7eac072f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:33.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.868+0000 7f7eb2356640 1 -- 172.21.15.71:0/912981534 >> v1:172.21.15.71:6789/0 conn(0x7f7eac14d9f0 legacy=0x7f7eac073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.868+0000 7f7eb2356640 1 -- 172.21.15.71:0/912981534 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac06c140 con 0x7f7eac0c3600 2024-04-16T15:41:33.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.868+0000 7f7eb1354640 1 -- 172.21.15.71:0/912981534 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344381898 0 0) 0x7f7eac06bb80 con 0x7f7eac14d9f0 2024-04-16T15:41:33.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb2356640 1 --2- 172.21.15.71:0/912981534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac0c3600 0x7f7eac153a20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009d90 tx=0x7f7ea402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2267b0d301b0347 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb1354640 1 -- 172.21.15.71:0/912981534 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4039070 con 0x7f7eac0c3600 2024-04-16T15:41:33.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb1354640 1 -- 172.21.15.71:0/912981534 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea402cb40 con 0x7f7eac0c3600 2024-04-16T15:41:33.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb1354640 1 -- 172.21.15.71:0/912981534 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4034440 con 0x7f7eac0c3600 2024-04-16T15:41:33.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb45ba640 1 -- 172.21.15.71:0/912981534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac0c3600 msgr2=0x7f7eac153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb45ba640 1 --2- 172.21.15.71:0/912981534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac0c3600 0x7f7eac153a20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009d90 tx=0x7f7ea402bd90 comp rx=0 tx=0).stop 2024-04-16T15:41:33.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb45ba640 1 -- 172.21.15.71:0/912981534 shutdown_connections 2024-04-16T15:41:33.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb45ba640 1 --2- 172.21.15.71:0/912981534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac0c3600 0x7f7eac153a20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb45ba640 1 -- 172.21.15.71:0/912981534 >> 172.21.15.71:0/912981534 conn(0x7f7eac06e220 msgr2=0x7f7eac0c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:33.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb45ba640 1 -- 172.21.15.71:0/912981534 shutdown_connections 2024-04-16T15:41:33.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.869+0000 7f7eb45ba640 1 -- 172.21.15.71:0/912981534 wait complete. 2024-04-16T15:41:33.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.870+0000 7f7eb45ba640 1 Processor -- start 2024-04-16T15:41:33.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb45ba640 1 -- start start 2024-04-16T15:41:33.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb45ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eac0c3600 0x7f7eac152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb45ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac14d540 0x7f7eac1eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb45ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7eac14d9f0 0x7f7eac1eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb45ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac14de20 con 0x7f7eac14d540 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb45ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac06d1f0 con 0x7f7eac0c3600 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb45ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac0c1d10 con 0x7f7eac14d9f0 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb1b55640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac14d540 0x7f7eac1eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb1b55640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac14d540 0x7f7eac1eb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53882/0 (socket says 172.21.15.71:53882) 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb1b55640 1 -- 172.21.15.71:0/1240625301 learned_addr learned my addr 172.21.15.71:0/1240625301 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.871+0000 7f7eb2356640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eac0c3600 0x7f7eac152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb2b57640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7eac14d9f0 0x7f7eac1eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb1b55640 1 -- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7eac14d9f0 msgr2=0x7f7eac1eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb1b55640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7eac14d9f0 0x7f7eac1eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb1b55640 1 -- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eac0c3600 msgr2=0x7f7eac152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb1b55640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eac0c3600 0x7f7eac152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb1b55640 1 -- 172.21.15.71:0/1240625301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac1ebf20 con 0x7f7eac14d540 2024-04-16T15:41:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb2b57640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7eac14d9f0 0x7f7eac1eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb2356640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eac0c3600 0x7f7eac152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb1b55640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac14d540 0x7f7eac1eb4a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c430 tx=0x7f7ea000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7e9effd640 1 -- 172.21.15.71:0/1240625301 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0011790 con 0x7f7eac14d540 2024-04-16T15:41:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7e9effd640 1 -- 172.21.15.71:0/1240625301 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea0017070 con 0x7f7eac14d540 2024-04-16T15:41:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7e9effd640 1 -- 172.21.15.71:0/1240625301 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0011bb0 con 0x7f7eac14d540 2024-04-16T15:41:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac1eafa0 con 0x7f7eac14d540 2024-04-16T15:41:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.872+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac148220 con 0x7f7eac14d540 2024-04-16T15:41:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.873+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eac148010 con 0x7f7eac14d540 2024-04-16T15:41:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.873+0000 7f7e9effd640 1 -- 172.21.15.71:0/1240625301 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ea001c460 con 0x7f7eac14d540 2024-04-16T15:41:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.874+0000 7f7e9effd640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7e9006c8f0 0x7f7e9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.874+0000 7f7e9effd640 1 -- 172.21.15.71:0/1240625301 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f7ea008bb70 con 0x7f7eac14d540 2024-04-16T15:41:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.874+0000 7f7eb2356640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7e9006c8f0 0x7f7e9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.874+0000 7f7eb2356640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7e9006c8f0 0x7f7e9006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4036000 tx=0x7f7ea4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:33.878+0000 7f7e9effd640 1 -- 172.21.15.71:0/1240625301 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7eac1ebf20 con 0x7f7eac14d540 2024-04-16T15:41:34.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.055+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f7eac1eb170 con 0x7f7e9006c8f0 2024-04-16T15:41:34.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.056+0000 7f7e9effd640 1 -- 172.21.15.71:0/1240625301 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7eac1eb170 con 0x7f7e9006c8f0 2024-04-16T15:41:34.056 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 4 to deep-scrub 2024-04-16T15:41:34.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7e9006c8f0 msgr2=0x7f7e9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7e9006c8f0 0x7f7e9006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4036000 tx=0x7f7ea4030000 comp rx=0 tx=0).stop 2024-04-16T15:41:34.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac14d540 msgr2=0x7f7eac1eb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac14d540 0x7f7eac1eb4a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000c430 tx=0x7f7ea000c8f0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 shutdown_connections 2024-04-16T15:41:34.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f7e9006c8f0 0x7f7e9006edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7eac14d9f0 0x7f7eac1eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7eac14d540 0x7f7eac1eb4a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 --2- 172.21.15.71:0/1240625301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eac0c3600 0x7f7eac152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 >> 172.21.15.71:0/1240625301 conn(0x7f7eac06e220 msgr2=0x7f7eac14bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 shutdown_connections 2024-04-16T15:41:34.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.058+0000 7f7eb45ba640 1 -- 172.21.15.71:0/1240625301 wait complete. 2024-04-16T15:41:34.068 INFO:tasks.ceph:Scrubbing osd.5 2024-04-16T15:41:34.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-16T15:41:34.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.177+0000 7f71d07a8640 1 Processor -- start 2024-04-16T15:41:34.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.177+0000 7f71d07a8640 1 -- start start 2024-04-16T15:41:34.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.178+0000 7f71d07a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c80cb0d0 0x7f71c80ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.178+0000 7f71d07a8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f71c80c7690 0x7f71c80ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.178+0000 7f71d07a8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c815a120 con 0x7f71c81533e0 2024-04-16T15:41:34.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.178+0000 7f71d07a8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f71c8156740 con 0x7f71c80c7690 2024-04-16T15:41:34.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.178+0000 7f71d07a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f71c81560f0 con 0x7f71c80cb0d0 2024-04-16T15:41:34.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.178+0000 7f71cdd43640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c80cb0d0 0x7f71c80ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.178+0000 7f71ced45640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f71c80c7690 0x7f71c80ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.178+0000 7f71ce544640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f71c81533e0 0x7f71c80ca460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.71:42190/0 (socket says 172.21.15.71:42190) 2024-04-16T15:41:34.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.179+0000 7f71ced45640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f71c80c7690 0x7f71c80ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.71:57024/0 (socket says 172.21.15.71:57024) 2024-04-16T15:41:34.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.179+0000 7f71cdd43640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c80cb0d0 0x7f71c80ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.71:54066/0 (socket says 172.21.15.71:54066) 2024-04-16T15:41:34.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.179+0000 7f71ce544640 1 -- 172.21.15.71:0/863405792 learned_addr learned my addr 172.21.15.71:0/863405792 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:34.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.179+0000 7f71cdd43640 1 -- 172.21.15.71:0/863405792 >> v2:172.21.15.122:3300/0 conn(0x7f71c80c7690 msgr2=0x7f71c80ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:34.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.179+0000 7f71cdd43640 1 --2- 172.21.15.71:0/863405792 >> v2:172.21.15.122:3300/0 conn(0x7f71c80c7690 0x7f71c80ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.179+0000 7f71cdd43640 1 -- 172.21.15.71:0/863405792 >> v1:172.21.15.122:6789/0 conn(0x7f71c81533e0 legacy=0x7f71c80ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.179+0000 7f71cdd43640 1 -- 172.21.15.71:0/863405792 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c81563d0 con 0x7f71c80cb0d0 2024-04-16T15:41:34.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.179+0000 7f71ced45640 1 --2- 172.21.15.71:0/863405792 >> v2:172.21.15.122:3300/0 conn(0x7f71c80c7690 0x7f71c80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T15:41:34.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.180+0000 7f71cdd43640 1 --2- 172.21.15.71:0/863405792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c80cb0d0 0x7f71c80ae3a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71b80097a0 tx=0x7f71b802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc234bffb22e645 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.180+0000 7f71cd542640 1 -- 172.21.15.71:0/863405792 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b80315a0 con 0x7f71c80cb0d0 2024-04-16T15:41:34.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.180+0000 7f71cd542640 1 -- 172.21.15.71:0/863405792 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b8037070 con 0x7f71c80cb0d0 2024-04-16T15:41:34.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.180+0000 7f71cd542640 1 -- 172.21.15.71:0/863405792 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8031960 con 0x7f71c80cb0d0 2024-04-16T15:41:34.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.181+0000 7f71d07a8640 1 -- 172.21.15.71:0/863405792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c80cb0d0 msgr2=0x7f71c80ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.181+0000 7f71d07a8640 1 --2- 172.21.15.71:0/863405792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c80cb0d0 0x7f71c80ae3a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71b80097a0 tx=0x7f71b802b840 comp rx=0 tx=0).stop 2024-04-16T15:41:34.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.181+0000 7f71d07a8640 1 -- 172.21.15.71:0/863405792 shutdown_connections 2024-04-16T15:41:34.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.181+0000 7f71d07a8640 1 --2- 172.21.15.71:0/863405792 >> v2:172.21.15.122:3300/0 conn(0x7f71c80c7690 0x7f71c80ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.181+0000 7f71d07a8640 1 --2- 172.21.15.71:0/863405792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c80cb0d0 0x7f71c80ae3a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.181+0000 7f71d07a8640 1 -- 172.21.15.71:0/863405792 >> 172.21.15.71:0/863405792 conn(0x7f71c80ad040 msgr2=0x7f71c80ad450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.181+0000 7f71d07a8640 1 -- 172.21.15.71:0/863405792 shutdown_connections 2024-04-16T15:41:34.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.181+0000 7f71d07a8640 1 -- 172.21.15.71:0/863405792 wait complete. 2024-04-16T15:41:34.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.182+0000 7f71d07a8640 1 Processor -- start 2024-04-16T15:41:34.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.182+0000 7f71d07a8640 1 -- start start 2024-04-16T15:41:34.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.182+0000 7f71d07a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71c80c7690 0x7f71c806e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.182+0000 7f71d07a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c80cb0d0 0x7f71c80c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71d07a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c81533e0 0x7f71c81ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71d07a8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8159d40 con 0x7f71c80cb0d0 2024-04-16T15:41:34.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71d07a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8155dc0 con 0x7f71c80c7690 2024-04-16T15:41:34.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71d07a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f71c8159ec0 con 0x7f71c81533e0 2024-04-16T15:41:34.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c80cb0d0 0x7f71c80c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c80cb0d0 0x7f71c80c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53890/0 (socket says 172.21.15.71:53890) 2024-04-16T15:41:34.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 -- 172.21.15.71:0/1919018283 learned_addr learned my addr 172.21.15.71:0/1919018283 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:34.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71ce544640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71c80c7690 0x7f71c806e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71ced45640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c81533e0 0x7f71c81ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 -- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c81533e0 msgr2=0x7f71c81ef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71c81533e0 0x7f71c81ef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 -- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71c80c7690 msgr2=0x7f71c806e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71c80c7690 0x7f71c806e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 -- 172.21.15.71:0/1919018283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c806d610 con 0x7f71c80cb0d0 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71cdd43640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c80cb0d0 0x7f71c80c5460 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f71b80097a0 tx=0x7f71b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8032630 con 0x7f71c80cb0d0 2024-04-16T15:41:34.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b80327d0 con 0x7f71c80cb0d0 2024-04-16T15:41:34.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c806d310 con 0x7f71c80cb0d0 2024-04-16T15:41:34.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8032bc0 con 0x7f71c80cb0d0 2024-04-16T15:41:34.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.183+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81eefb0 con 0x7f71c80cb0d0 2024-04-16T15:41:34.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.184+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71c81ef390 con 0x7f71c80cb0d0 2024-04-16T15:41:34.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.185+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71b804d3f0 con 0x7f71c80cb0d0 2024-04-16T15:41:34.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.185+0000 7f71beffd640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f71ac06c8f0 0x7f71ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.185+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f71b802f030 con 0x7f71c80cb0d0 2024-04-16T15:41:34.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.185+0000 7f71ce544640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f71ac06c8f0 0x7f71ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.185+0000 7f71beffd640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7f71ac075160 0x7f71ac0775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.185+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 --> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71b803ce50 con 0x7f71ac075160 2024-04-16T15:41:34.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.185+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f71b80b4050 con 0x7f71c80cb0d0 2024-04-16T15:41:34.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.186+0000 7f71ced45640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7f71ac075160 0x7f71ac0775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.186+0000 7f71ce544640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f71ac06c8f0 0x7f71ac06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f71c40012e0 tx=0x7f71c4001990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.186+0000 7f71ced45640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7f71ac075160 0x7f71ac0775a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.187+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 <== osd.5 v2:172.21.15.122:6817/1133463708 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71b803ce50 con 0x7f71ac075160 2024-04-16T15:41:34.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.200+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 --> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f71c8068be0 con 0x7f71ac075160 2024-04-16T15:41:34.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.200+0000 7f71beffd640 1 -- 172.21.15.71:0/1919018283 <== osd.5 v2:172.21.15.122:6817/1133463708 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f71c8068be0 con 0x7f71ac075160 2024-04-16T15:41:34.200 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:34.200 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T15:41:34.201 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:34.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.200+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7f71ac075160 msgr2=0x7f71ac0775a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.200+0000 7f71d07a8640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.122:6817/1133463708,v1:172.21.15.122:6819/1133463708] conn(0x7f71ac075160 0x7f71ac0775a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.200+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f71ac06c8f0 msgr2=0x7f71ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.201+0000 7f71d07a8640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f71ac06c8f0 0x7f71ac06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f71c40012e0 tx=0x7f71c4001990 comp rx=0 tx=0).stop 2024-04-16T15:41:34.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.201+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c80cb0d0 msgr2=0x7f71c80c5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.201+0000 7f71d07a8640 1 --2- 172.21.15.71:0/1919018283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71c80cb0d0 0x7f71c80c5460 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f71b80097a0 tx=0x7f71b80046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.201+0000 7f71ced45640 1 -- 172.21.15.71:0/1919018283 reap_dead start 2024-04-16T15:41:34.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.201+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 shutdown_connections 2024-04-16T15:41:34.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.201+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 >> 172.21.15.71:0/1919018283 conn(0x7f71c80ad040 msgr2=0x7f71c80ca180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.201+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 shutdown_connections 2024-04-16T15:41:34.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.201+0000 7f71d07a8640 1 -- 172.21.15.71:0/1919018283 wait complete. 2024-04-16T15:41:34.211 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-16T15:41:34.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.315+0000 7f11b9035640 1 Processor -- start 2024-04-16T15:41:34.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.315+0000 7f11b9035640 1 -- start start 2024-04-16T15:41:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.315+0000 7f11b9035640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f11b414a5d0 0x7f11b406da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.315+0000 7f11b9035640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f11b406f8a0 0x7f11b41539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.315+0000 7f11b9035640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11b40a9780 con 0x7f11b406df90 2024-04-16T15:41:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.315+0000 7f11b9035640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f11b40a94a0 con 0x7f11b406f8a0 2024-04-16T15:41:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.315+0000 7f11b9035640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f11b4157a50 con 0x7f11b414a5d0 2024-04-16T15:41:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.316+0000 7f11b259c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f11b406f8a0 0x7f11b41539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.316+0000 7f11b359e640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f11b406df90 0x7f11b4153ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35114/0 (socket says 172.21.15.71:35114) 2024-04-16T15:41:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.316+0000 7f11b359e640 1 -- 172.21.15.71:0/210555445 learned_addr learned my addr 172.21.15.71:0/210555445 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.316+0000 7f11b259c640 1 -- 172.21.15.71:0/210555445 >> v2:172.21.15.122:3300/0 conn(0x7f11b414a5d0 msgr2=0x7f11b406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.316+0000 7f11b259c640 1 --2- 172.21.15.71:0/210555445 >> v2:172.21.15.122:3300/0 conn(0x7f11b414a5d0 0x7f11b406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.316+0000 7f11b259c640 1 -- 172.21.15.71:0/210555445 >> v1:172.21.15.71:6789/0 conn(0x7f11b406df90 legacy=0x7f11b4153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.316+0000 7f11b259c640 1 -- 172.21.15.71:0/210555445 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b4157d30 con 0x7f11b406f8a0 2024-04-16T15:41:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.316+0000 7f11b1d9b640 1 -- 172.21.15.71:0/210555445 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625427600 0 0) 0x7f11b40a9780 con 0x7f11b406df90 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b259c640 1 --2- 172.21.15.71:0/210555445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406f8a0 0x7f11b41539b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f11b40741b0 tx=0x7f119c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e6ba94302d4c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b1d9b640 1 -- 172.21.15.71:0/210555445 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c035020 con 0x7f11b406f8a0 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b1d9b640 1 -- 172.21.15.71:0/210555445 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c031830 con 0x7f11b406f8a0 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b1d9b640 1 -- 172.21.15.71:0/210555445 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c031c00 con 0x7f11b406f8a0 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b9035640 1 -- 172.21.15.71:0/210555445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406f8a0 msgr2=0x7f11b41539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b9035640 1 --2- 172.21.15.71:0/210555445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406f8a0 0x7f11b41539b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f11b40741b0 tx=0x7f119c02b8d0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b9035640 1 -- 172.21.15.71:0/210555445 shutdown_connections 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b9035640 1 --2- 172.21.15.71:0/210555445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406f8a0 0x7f11b41539b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b9035640 1 --2- 172.21.15.71:0/210555445 >> v2:172.21.15.122:3300/0 conn(0x7f11b414a5d0 0x7f11b406da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b9035640 1 -- 172.21.15.71:0/210555445 >> 172.21.15.71:0/210555445 conn(0x7f11b40729d0 msgr2=0x7f11b4072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b9035640 1 -- 172.21.15.71:0/210555445 shutdown_connections 2024-04-16T15:41:34.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.317+0000 7f11b9035640 1 -- 172.21.15.71:0/210555445 wait complete. 2024-04-16T15:41:34.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.319+0000 7f11b9035640 1 Processor -- start 2024-04-16T15:41:34.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.319+0000 7f11b9035640 1 -- start start 2024-04-16T15:41:34.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.319+0000 7f11b9035640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406df90 0x7f11b414e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.319+0000 7f11b9035640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11b406f8a0 0x7f11b41ed550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.319+0000 7f11b9035640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11b414a5d0 0x7f11b4156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.319+0000 7f11b9035640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11b40ae0e0 con 0x7f11b406df90 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.319+0000 7f11b9035640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11b40adaa0 con 0x7f11b406f8a0 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.319+0000 7f11b9035640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f11b4154600 con 0x7f11b414a5d0 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406df90 0x7f11b414e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406df90 0x7f11b414e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53904/0 (socket says 172.21.15.71:53904) 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 -- 172.21.15.71:0/77898711 learned_addr learned my addr 172.21.15.71:0/77898711 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b359e640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11b414a5d0 0x7f11b4156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b259c640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11b406f8a0 0x7f11b41ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 -- 172.21.15.71:0/77898711 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11b414a5d0 msgr2=0x7f11b4156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11b414a5d0 0x7f11b4156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 -- 172.21.15.71:0/77898711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11b406f8a0 msgr2=0x7f11b41ed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11b406f8a0 0x7f11b41ed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 -- 172.21.15.71:0/77898711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b41e95d0 con 0x7f11b406df90 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b2d9d640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406df90 0x7f11b414e280 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00b410 tx=0x7f11ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11ab7fe640 1 -- 172.21.15.71:0/77898711 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac015020 con 0x7f11b406df90 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11ab7fe640 1 -- 172.21.15.71:0/77898711 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac011850 con 0x7f11b406df90 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11ab7fe640 1 -- 172.21.15.71:0/77898711 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac011c40 con 0x7f11b406df90 2024-04-16T15:41:34.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b40c78d0 con 0x7f11b406df90 2024-04-16T15:41:34.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.320+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b40c7e90 con 0x7f11b406df90 2024-04-16T15:41:34.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.321+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b4068c10 con 0x7f11b406df90 2024-04-16T15:41:34.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.322+0000 7f11ab7fe640 1 -- 172.21.15.71:0/77898711 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11ac018020 con 0x7f11b406df90 2024-04-16T15:41:34.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.322+0000 7f11ab7fe640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f119406c8f0 0x7f119406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.322+0000 7f11b259c640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f119406c8f0 0x7f119406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.322+0000 7f11ab7fe640 1 -- 172.21.15.71:0/77898711 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f11ac08bb30 con 0x7f11b406df90 2024-04-16T15:41:34.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.323+0000 7f11b259c640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f119406c8f0 0x7f119406edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f119c036000 tx=0x7f119c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.327+0000 7f11ab7fe640 1 -- 172.21.15.71:0/77898711 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11ac01f020 con 0x7f11b406df90 2024-04-16T15:41:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.500+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f11b414de30 con 0x7f119406c8f0 2024-04-16T15:41:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.500+0000 7f11ab7fe640 1 -- 172.21.15.71:0/77898711 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f11b414de30 con 0x7f119406c8f0 2024-04-16T15:41:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 5 to deep-scrub 2024-04-16T15:41:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f119406c8f0 msgr2=0x7f119406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f119406c8f0 0x7f119406edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f119c036000 tx=0x7f119c03c040 comp rx=0 tx=0).stop 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406df90 msgr2=0x7f11b414e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406df90 0x7f11b414e280 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00b410 tx=0x7f11ac00b8d0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 shutdown_connections 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f119406c8f0 0x7f119406edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11b414a5d0 0x7f11b4156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11b406f8a0 0x7f11b41ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 --2- 172.21.15.71:0/77898711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b406df90 0x7f11b414e280 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 >> 172.21.15.71:0/77898711 conn(0x7f11b40729d0 msgr2=0x7f11b4149310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.502+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 shutdown_connections 2024-04-16T15:41:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.503+0000 7f11b9035640 1 -- 172.21.15.71:0/77898711 wait complete. 2024-04-16T15:41:34.513 INFO:tasks.ceph:Scrubbing osd.6 2024-04-16T15:41:34.513 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-16T15:41:34.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.621+0000 7f29ebf15640 1 Processor -- start 2024-04-16T15:41:34.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.622+0000 7f29ebf15640 1 -- start start 2024-04-16T15:41:34.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.622+0000 7f29ebf15640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f29e414c910 0x7f29e40c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.622+0000 7f29ebf15640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f29e406bb50 0x7f29e4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.622+0000 7f29ebf15640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e4156b00 con 0x7f29e4155fe0 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.622+0000 7f29ebf15640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f29e40c3160 con 0x7f29e406bb50 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.622+0000 7f29ebf15640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f29e4156dd0 con 0x7f29e414c910 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29e94b0640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f29e406bb50 0x7f29e4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29ea4b2640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f29e4155fe0 0x7f29e41563f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35126/0 (socket says 172.21.15.71:35126) 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29e94b0640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f29e406bb50 0x7f29e4071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53916/0 (socket says 172.21.15.71:53916) 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29ea4b2640 1 -- 172.21.15.71:0/131328340 learned_addr learned my addr 172.21.15.71:0/131328340 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29e94b0640 1 -- 172.21.15.71:0/131328340 >> v2:172.21.15.122:3300/0 conn(0x7f29e414c910 msgr2=0x7f29e40c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29e94b0640 1 --2- 172.21.15.71:0/131328340 >> v2:172.21.15.122:3300/0 conn(0x7f29e414c910 0x7f29e40c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29e94b0640 1 -- 172.21.15.71:0/131328340 >> v1:172.21.15.71:6789/0 conn(0x7f29e4155fe0 legacy=0x7f29e41563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29e94b0640 1 -- 172.21.15.71:0/131328340 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e40c2830 con 0x7f29e406bb50 2024-04-16T15:41:34.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.623+0000 7f29e8caf640 1 -- 172.21.15.71:0/131328340 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437589717 0 0) 0x7f29e4156b00 con 0x7f29e4155fe0 2024-04-16T15:41:34.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29e94b0640 1 --2- 172.21.15.71:0/131328340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 0x7f29e4071d00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f29e40c1c50 tx=0x7f29d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf55dba6de4ed5c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29e8caf640 1 -- 172.21.15.71:0/131328340 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4035020 con 0x7f29e406bb50 2024-04-16T15:41:34.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29e8caf640 1 -- 172.21.15.71:0/131328340 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d4031830 con 0x7f29e406bb50 2024-04-16T15:41:34.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29e8caf640 1 -- 172.21.15.71:0/131328340 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4031c00 con 0x7f29e406bb50 2024-04-16T15:41:34.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29ebf15640 1 -- 172.21.15.71:0/131328340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 msgr2=0x7f29e4071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29ebf15640 1 --2- 172.21.15.71:0/131328340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 0x7f29e4071d00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f29e40c1c50 tx=0x7f29d402b8d0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29ebf15640 1 -- 172.21.15.71:0/131328340 shutdown_connections 2024-04-16T15:41:34.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29ebf15640 1 --2- 172.21.15.71:0/131328340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 0x7f29e4071d00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29ebf15640 1 --2- 172.21.15.71:0/131328340 >> v2:172.21.15.122:3300/0 conn(0x7f29e414c910 0x7f29e40c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29ebf15640 1 -- 172.21.15.71:0/131328340 >> 172.21.15.71:0/131328340 conn(0x7f29e40c3e80 msgr2=0x7f29e40c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29ebf15640 1 -- 172.21.15.71:0/131328340 shutdown_connections 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.624+0000 7f29ebf15640 1 -- 172.21.15.71:0/131328340 wait complete. 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.625+0000 7f29ebf15640 1 Processor -- start 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29ebf15640 1 -- start start 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29ebf15640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 0x7f29e40ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29ebf15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29e414c910 0x7f29e413a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29ebf15640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f29e4155fe0 0x7f29e413af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29ebf15640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4071930 con 0x7f29e406bb50 2024-04-16T15:41:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29ebf15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40c2df0 con 0x7f29e414c910 2024-04-16T15:41:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29ebf15640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f29e4071ab0 con 0x7f29e4155fe0 2024-04-16T15:41:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29e9cb1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 0x7f29e40ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29e9cb1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 0x7f29e40ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53922/0 (socket says 172.21.15.71:53922) 2024-04-16T15:41:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29e9cb1640 1 -- 172.21.15.71:0/173010654 learned_addr learned my addr 172.21.15.71:0/173010654 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29e94b0640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29e414c910 0x7f29e413a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.626+0000 7f29ea4b2640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f29e4155fe0 0x7f29e413af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29e9cb1640 1 -- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f29e4155fe0 msgr2=0x7f29e413af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29e9cb1640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f29e4155fe0 0x7f29e413af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29e9cb1640 1 -- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29e414c910 msgr2=0x7f29e413a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29e9cb1640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29e414c910 0x7f29e413a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29e9cb1640 1 -- 172.21.15.71:0/173010654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e40c08b0 con 0x7f29e406bb50 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29ea4b2640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f29e4155fe0 0x7f29e413af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29e9cb1640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 0x7f29e40ac720 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f29e000b4a0 tx=0x7f29e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0014070 con 0x7f29e406bb50 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e0010610 con 0x7f29e406bb50 2024-04-16T15:41:34.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e4153e70 con 0x7f29e406bb50 2024-04-16T15:41:34.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0010a00 con 0x7f29e406bb50 2024-04-16T15:41:34.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.627+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e4154430 con 0x7f29e406bb50 2024-04-16T15:41:34.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.628+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29e4154810 con 0x7f29e406bb50 2024-04-16T15:41:34.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.628+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29e0017070 con 0x7f29e406bb50 2024-04-16T15:41:34.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.629+0000 7f29da7fc640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f29c806c8f0 0x7f29c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.629+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f29e008b6a0 con 0x7f29e406bb50 2024-04-16T15:41:34.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.629+0000 7f29e94b0640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f29c806c8f0 0x7f29c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.629+0000 7f29da7fc640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7f29c8075160 0x7f29c80775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.629+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 --> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29e0021e50 con 0x7f29c8075160 2024-04-16T15:41:34.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.629+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f29e0092050 con 0x7f29e406bb50 2024-04-16T15:41:34.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.629+0000 7f29ea4b2640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7f29c8075160 0x7f29c80775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.629+0000 7f29e94b0640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f29c806c8f0 0x7f29c806edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29d4036000 tx=0x7f29d403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.630+0000 7f29ea4b2640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7f29c8075160 0x7f29c80775a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.630+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 <== osd.6 v2:172.21.15.122:6808/1958778321 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f29e0021e50 con 0x7f29c8075160 2024-04-16T15:41:34.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.643+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 --> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f29e4072e60 con 0x7f29c8075160 2024-04-16T15:41:34.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.644+0000 7f29da7fc640 1 -- 172.21.15.71:0/173010654 <== osd.6 v2:172.21.15.122:6808/1958778321 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f29e4072e60 con 0x7f29c8075160 2024-04-16T15:41:34.644 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:34.644 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T15:41:34.644 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:34.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.644+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7f29c8075160 msgr2=0x7f29c80775a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.644+0000 7f29ebf15640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.122:6808/1958778321,v1:172.21.15.122:6809/1958778321] conn(0x7f29c8075160 0x7f29c80775a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.644+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f29c806c8f0 msgr2=0x7f29c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.644+0000 7f29ebf15640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f29c806c8f0 0x7f29c806edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29d4036000 tx=0x7f29d403c040 comp rx=0 tx=0).stop 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.644+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 msgr2=0x7f29e40ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.644+0000 7f29ebf15640 1 --2- 172.21.15.71:0/173010654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29e406bb50 0x7f29e40ac720 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f29e000b4a0 tx=0x7f29e000b960 comp rx=0 tx=0).stop 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.644+0000 7f29ea4b2640 1 -- 172.21.15.71:0/173010654 reap_dead start 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.645+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 shutdown_connections 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.645+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 >> 172.21.15.71:0/173010654 conn(0x7f29e40c3e80 msgr2=0x7f29e40c1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.645+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 shutdown_connections 2024-04-16T15:41:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.645+0000 7f29ebf15640 1 -- 172.21.15.71:0/173010654 wait complete. 2024-04-16T15:41:34.654 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-16T15:41:34.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.758+0000 7f5b93c74640 1 Processor -- start 2024-04-16T15:41:34.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b93c74640 1 -- start start 2024-04-16T15:41:34.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b93c74640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5b8c0a85e0 0x7f5b8c071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b93c74640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5b8c06e5b0 0x7f5b8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b93c74640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c06e9e0 con 0x7f5b8c06e1d0 2024-04-16T15:41:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b93c74640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c155f80 con 0x7f5b8c0a85e0 2024-04-16T15:41:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b93c74640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c156260 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b9120f640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f5b8c0a85e0 0x7f5b8c071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b91a10640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f5b8c06e1d0 0x7f5b8c06ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35136/0 (socket says 172.21.15.71:35136) 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.759+0000 7f5b91a10640 1 -- 172.21.15.71:0/3776859306 learned_addr learned my addr 172.21.15.71:0/3776859306 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.760+0000 7f5b92211640 1 --2- 172.21.15.71:0/3776859306 >> v2:172.21.15.122:3300/0 conn(0x7f5b8c06e5b0 0x7f5b8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.760+0000 7f5b9120f640 1 -- 172.21.15.71:0/3776859306 >> v2:172.21.15.122:3300/0 conn(0x7f5b8c06e5b0 msgr2=0x7f5b8c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.760+0000 7f5b9120f640 1 --2- 172.21.15.71:0/3776859306 >> v2:172.21.15.122:3300/0 conn(0x7f5b8c06e5b0 0x7f5b8c06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.760+0000 7f5b9120f640 1 -- 172.21.15.71:0/3776859306 >> v1:172.21.15.71:6789/0 conn(0x7f5b8c06e1d0 legacy=0x7f5b8c06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.760+0000 7f5b9120f640 1 -- 172.21.15.71:0/3776859306 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c156540 con 0x7f5b8c0a85e0 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.760+0000 7f5b90a0e640 1 -- 172.21.15.71:0/3776859306 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588853216 0 0) 0x7f5b8c06e9e0 con 0x7f5b8c06e1d0 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.760+0000 7f5b92211640 1 --2- 172.21.15.71:0/3776859306 >> v2:172.21.15.122:3300/0 conn(0x7f5b8c06e5b0 0x7f5b8c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b9120f640 1 --2- 172.21.15.71:0/3776859306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c0a85e0 0x7f5b8c071b00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0634a0 tx=0x7f5b7c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69fc97c2e4e0e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b90a0e640 1 -- 172.21.15.71:0/3776859306 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c038070 con 0x7f5b8c0a85e0 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b90a0e640 1 -- 172.21.15.71:0/3776859306 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c031710 con 0x7f5b8c0a85e0 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b90a0e640 1 -- 172.21.15.71:0/3776859306 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c031ae0 con 0x7f5b8c0a85e0 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b93c74640 1 -- 172.21.15.71:0/3776859306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c0a85e0 msgr2=0x7f5b8c071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b93c74640 1 --2- 172.21.15.71:0/3776859306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c0a85e0 0x7f5b8c071b00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0634a0 tx=0x7f5b7c02b7b0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b93c74640 1 -- 172.21.15.71:0/3776859306 shutdown_connections 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b93c74640 1 --2- 172.21.15.71:0/3776859306 >> v2:172.21.15.122:3300/0 conn(0x7f5b8c06e5b0 0x7f5b8c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b93c74640 1 --2- 172.21.15.71:0/3776859306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c0a85e0 0x7f5b8c071b00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b93c74640 1 -- 172.21.15.71:0/3776859306 >> 172.21.15.71:0/3776859306 conn(0x7f5b8c06ee30 msgr2=0x7f5b8c06f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b93c74640 1 -- 172.21.15.71:0/3776859306 shutdown_connections 2024-04-16T15:41:34.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.761+0000 7f5b93c74640 1 -- 172.21.15.71:0/3776859306 wait complete. 2024-04-16T15:41:34.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.762+0000 7f5b93c74640 1 Processor -- start 2024-04-16T15:41:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.762+0000 7f5b93c74640 1 -- start start 2024-04-16T15:41:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b93c74640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c06e1d0 0x7f5b8c154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b93c74640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c06e5b0 0x7f5b8c14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b93c74640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b8c0a85e0 0x7f5b8c14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b93c74640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c06d690 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b93c74640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0a8bc0 con 0x7f5b8c06e1d0 2024-04-16T15:41:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b93c74640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c155c30 con 0x7f5b8c0a85e0 2024-04-16T15:41:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b9120f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c06e5b0 0x7f5b8c14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b9120f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c06e5b0 0x7f5b8c14df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53934/0 (socket says 172.21.15.71:53934) 2024-04-16T15:41:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b9120f640 1 -- 172.21.15.71:0/4195669737 learned_addr learned my addr 172.21.15.71:0/4195669737 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b9120f640 1 -- 172.21.15.71:0/4195669737 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b8c0a85e0 msgr2=0x7f5b8c14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.763+0000 7f5b9120f640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b8c0a85e0 0x7f5b8c14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b9120f640 1 -- 172.21.15.71:0/4195669737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c06e1d0 msgr2=0x7f5b8c154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b91a10640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c06e1d0 0x7f5b8c154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b9120f640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c06e1d0 0x7f5b8c154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b9120f640 1 -- 172.21.15.71:0/4195669737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c0c8980 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b9120f640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c06e5b0 0x7f5b8c14df10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c000c00 tx=0x7f5b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b827fc640 1 -- 172.21.15.71:0/4195669737 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c042070 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b827fc640 1 -- 172.21.15.71:0/4195669737 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c033a40 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c0c8c60 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b827fc640 1 -- 172.21.15.71:0/4195669737 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c038040 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.764+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c1501b0 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.765+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c068be0 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.765+0000 7f5b827fc640 1 -- 172.21.15.71:0/4195669737 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b7c03b070 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.766+0000 7f5b827fc640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b7006c920 0x7f5b7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.766+0000 7f5b827fc640 1 -- 172.21.15.71:0/4195669737 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f5b7c0ad610 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.766+0000 7f5b91a10640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b7006c920 0x7f5b7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.766+0000 7f5b91a10640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b7006c920 0x7f5b7006ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b88000f60 tx=0x7f5b88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.770+0000 7f5b827fc640 1 -- 172.21.15.71:0/4195669737 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b7c07b8b0 con 0x7f5b8c06e5b0 2024-04-16T15:41:34.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.948+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f5b8c0c8f40 con 0x7f5b7006c920 2024-04-16T15:41:34.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.948+0000 7f5b827fc640 1 -- 172.21.15.71:0/4195669737 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5b8c0c8f40 con 0x7f5b7006c920 2024-04-16T15:41:34.948 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 6 to deep-scrub 2024-04-16T15:41:34.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b7006c920 msgr2=0x7f5b7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b7006c920 0x7f5b7006ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b88000f60 tx=0x7f5b88009040 comp rx=0 tx=0).stop 2024-04-16T15:41:34.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c06e5b0 msgr2=0x7f5b8c14df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:34.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c06e5b0 0x7f5b8c14df10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c000c00 tx=0x7f5b7c0046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 shutdown_connections 2024-04-16T15:41:34.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f5b7006c920 0x7f5b7006ede0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5b8c0a85e0 0x7f5b8c14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b8c06e5b0 0x7f5b8c14df10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 --2- 172.21.15.71:0/4195669737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c06e1d0 0x7f5b8c154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:34.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 >> 172.21.15.71:0/4195669737 conn(0x7f5b8c06ee30 msgr2=0x7f5b8c155090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:34.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 shutdown_connections 2024-04-16T15:41:34.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:34.950+0000 7f5b93c74640 1 -- 172.21.15.71:0/4195669737 wait complete. 2024-04-16T15:41:34.961 INFO:tasks.ceph:Scrubbing osd.7 2024-04-16T15:41:34.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-16T15:41:35.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.068+0000 7f612d38f640 1 Processor -- start 2024-04-16T15:41:35.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.068+0000 7f612d38f640 1 -- start start 2024-04-16T15:41:35.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f612d38f640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f61280a8640 0x7f61280bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f612d38f640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61280a8db0 con 0x7f612814fed0 2024-04-16T15:41:35.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f612d38f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61280bff50 con 0x7f6128156740 2024-04-16T15:41:35.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f612d38f640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f61280bfd20 con 0x7f61280a8640 2024-04-16T15:41:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f61267fc640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f61280a8640 0x7f61280bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f6126ffd640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f612814fed0 0x7f61280c72a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:35140/0 (socket says 172.21.15.71:35140) 2024-04-16T15:41:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f6126ffd640 1 -- 172.21.15.71:0/3680023417 learned_addr learned my addr 172.21.15.71:0/3680023417 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f61267fc640 1 -- 172.21.15.71:0/3680023417 >> v1:172.21.15.122:6789/0 conn(0x7f6128156740 legacy=0x7f61280c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f61267fc640 1 -- 172.21.15.71:0/3680023417 >> v1:172.21.15.71:6789/0 conn(0x7f612814fed0 legacy=0x7f61280c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f61267fc640 1 -- 172.21.15.71:0/3680023417 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61280c0230 con 0x7f61280a8640 2024-04-16T15:41:35.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.069+0000 7f6125ffb640 1 -- 172.21.15.71:0/3680023417 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 779378535 0 0) 0x7f6118001490 con 0x7f612814fed0 2024-04-16T15:41:35.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.070+0000 7f61267fc640 1 --2- 172.21.15.71:0/3680023417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61280a8640 0x7f61280bf7e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f611c0097a0 tx=0x7f611c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432defa8ecebcbb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.070+0000 7f6125ffb640 1 -- 172.21.15.71:0/3680023417 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c035020 con 0x7f61280a8640 2024-04-16T15:41:35.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.070+0000 7f6125ffb640 1 -- 172.21.15.71:0/3680023417 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c031830 con 0x7f61280a8640 2024-04-16T15:41:35.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.070+0000 7f6125ffb640 1 -- 172.21.15.71:0/3680023417 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c031c00 con 0x7f61280a8640 2024-04-16T15:41:35.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.071+0000 7f612d38f640 1 -- 172.21.15.71:0/3680023417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61280a8640 msgr2=0x7f61280bf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.071+0000 7f612d38f640 1 --2- 172.21.15.71:0/3680023417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61280a8640 0x7f61280bf7e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f611c0097a0 tx=0x7f611c02b8d0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.071+0000 7f612d38f640 1 -- 172.21.15.71:0/3680023417 shutdown_connections 2024-04-16T15:41:35.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.071+0000 7f612d38f640 1 --2- 172.21.15.71:0/3680023417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61280a8640 0x7f61280bf7e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.071+0000 7f612d38f640 1 -- 172.21.15.71:0/3680023417 >> 172.21.15.71:0/3680023417 conn(0x7f61280c5e20 msgr2=0x7f61280c6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:35.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.071+0000 7f612d38f640 1 -- 172.21.15.71:0/3680023417 shutdown_connections 2024-04-16T15:41:35.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.071+0000 7f612d38f640 1 -- 172.21.15.71:0/3680023417 wait complete. 2024-04-16T15:41:35.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.072+0000 7f612d38f640 1 Processor -- start 2024-04-16T15:41:35.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.072+0000 7f612d38f640 1 -- start start 2024-04-16T15:41:35.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f612d38f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61280a8640 0x7f61280ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f612d38f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f612814fed0 0x7f6128154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f612d38f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6128156740 0x7f6128149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f612d38f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6128156220 con 0x7f6128156740 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f612d38f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f61280c0510 con 0x7f612814fed0 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f612d38f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f61280c8d50 con 0x7f61280a8640 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6128156740 0x7f6128149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6128156740 0x7f6128149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53946/0 (socket says 172.21.15.71:53946) 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 -- 172.21.15.71:0/2429875376 learned_addr learned my addr 172.21.15.71:0/2429875376 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f6126ffd640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61280a8640 0x7f61280ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61267fc640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f612814fed0 0x7f6128154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 -- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61280a8640 msgr2=0x7f61280ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f61280a8640 0x7f61280ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 -- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f612814fed0 msgr2=0x7f6128154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f612814fed0 0x7f6128154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 -- 172.21.15.71:0/2429875376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61280c3800 con 0x7f6128156740 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.073+0000 7f61277fe640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6128156740 0x7f6128149540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f612000c3c0 tx=0x7f612000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.074+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120017070 con 0x7f6128156740 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.074+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61200035f0 con 0x7f6128156740 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.074+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61280ad930 con 0x7f6128156740 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.074+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200039e0 con 0x7f6128156740 2024-04-16T15:41:35.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.074+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61280c3240 con 0x7f6128156740 2024-04-16T15:41:35.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.075+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6128068be0 con 0x7f6128156740 2024-04-16T15:41:35.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.075+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f612001a070 con 0x7f6128156740 2024-04-16T15:41:35.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.076+0000 7f610f7fe640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f611406c8f0 0x7f611406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.076+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f612008b0d0 con 0x7f6128156740 2024-04-16T15:41:35.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.076+0000 7f6126ffd640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f611406c8f0 0x7f611406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.076+0000 7f610f7fe640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f6114075160 0x7f61140775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.076+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 --> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6120015e70 con 0x7f6114075160 2024-04-16T15:41:35.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.076+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6120092050 con 0x7f6128156740 2024-04-16T15:41:35.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.076+0000 7f6126ffd640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f611406c8f0 0x7f611406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61180012e0 tx=0x7f6118007cf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.076+0000 7f61267fc640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f6114075160 0x7f61140775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.077+0000 7f61267fc640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f6114075160 0x7f61140775a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.077+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 <== osd.7 v2:172.21.15.122:6800/3651959178 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6120015e70 con 0x7f6114075160 2024-04-16T15:41:35.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.091+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 --> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f61280c8420 con 0x7f6114075160 2024-04-16T15:41:35.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.091+0000 7f610f7fe640 1 -- 172.21.15.71:0/2429875376 <== osd.7 v2:172.21.15.122:6800/3651959178 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f61280c8420 con 0x7f6114075160 2024-04-16T15:41:35.091 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-16T15:41:35.092 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-16T15:41:35.092 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-16T15:41:35.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.091+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f6114075160 msgr2=0x7f61140775a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.122:6800/3651959178,v1:172.21.15.122:6801/3651959178] conn(0x7f6114075160 0x7f61140775a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f611406c8f0 msgr2=0x7f611406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f611406c8f0 0x7f611406edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61180012e0 tx=0x7f6118007cf0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6128156740 msgr2=0x7f6128149540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 --2- 172.21.15.71:0/2429875376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6128156740 0x7f6128149540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f612000c3c0 tx=0x7f612000c880 comp rx=0 tx=0).stop 2024-04-16T15:41:35.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f61277fe640 1 -- 172.21.15.71:0/2429875376 reap_dead start 2024-04-16T15:41:35.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 shutdown_connections 2024-04-16T15:41:35.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 >> 172.21.15.71:0/2429875376 conn(0x7f61280c5e20 msgr2=0x7f612814ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:35.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 shutdown_connections 2024-04-16T15:41:35.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.092+0000 7f612d38f640 1 -- 172.21.15.71:0/2429875376 wait complete. 2024-04-16T15:41:35.102 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-16T15:41:35.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.206+0000 7f2028655640 1 Processor -- start 2024-04-16T15:41:35.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.206+0000 7f2028655640 1 -- start start 2024-04-16T15:41:35.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2028655640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20200c3600 0x7f2020153a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2028655640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f202014d540 0x7f202014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2028655640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2020072f10 0x7f2020073380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2028655640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f202006b930 con 0x7f2020072f10 2024-04-16T15:41:35.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2028655640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f202014df00 con 0x7f202014d540 2024-04-16T15:41:35.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2028655640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f202006bb80 con 0x7f20200c3600 2024-04-16T15:41:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2026bf2640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2020072f10 0x7f2020073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2026bf2640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f2020072f10 0x7f2020073380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53950/0 (socket says 172.21.15.71:53950) 2024-04-16T15:41:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2026bf2640 1 -- 172.21.15.71:0/4263456840 learned_addr learned my addr 172.21.15.71:0/4263456840 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f20263f1640 1 --2- 172.21.15.71:0/4263456840 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20200c3600 0x7f2020153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2025bf0640 1 --2- 172.21.15.71:0/4263456840 >> v2:172.21.15.122:3300/0 conn(0x7f202014d540 0x7f202014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2026bf2640 1 -- 172.21.15.71:0/4263456840 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20200c3600 msgr2=0x7f2020153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:35.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2026bf2640 1 --2- 172.21.15.71:0/4263456840 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20200c3600 0x7f2020153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2026bf2640 1 -- 172.21.15.71:0/4263456840 >> v2:172.21.15.122:3300/0 conn(0x7f202014d540 msgr2=0x7f202014d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:35.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2026bf2640 1 --2- 172.21.15.71:0/4263456840 >> v2:172.21.15.122:3300/0 conn(0x7f202014d540 0x7f202014d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.207+0000 7f2026bf2640 1 -- 172.21.15.71:0/4263456840 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202006be60 con 0x7f2020072f10 2024-04-16T15:41:35.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.208+0000 7f2026bf2640 1 --2- 172.21.15.71:0/4263456840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020072f10 0x7f2020073380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f20200ac840 tx=0x7f201802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36aabc392baf62b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.208+0000 7f20253ef640 1 -- 172.21.15.71:0/4263456840 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201802c950 con 0x7f2020072f10 2024-04-16T15:41:35.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.208+0000 7f20253ef640 1 -- 172.21.15.71:0/4263456840 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201802caf0 con 0x7f2020072f10 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.208+0000 7f20253ef640 1 -- 172.21.15.71:0/4263456840 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180343f0 con 0x7f2020072f10 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 -- 172.21.15.71:0/4263456840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020072f10 msgr2=0x7f2020073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 --2- 172.21.15.71:0/4263456840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020072f10 0x7f2020073380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f20200ac840 tx=0x7f201802bdb0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 -- 172.21.15.71:0/4263456840 shutdown_connections 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 --2- 172.21.15.71:0/4263456840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020072f10 0x7f2020073380 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 --2- 172.21.15.71:0/4263456840 >> v2:172.21.15.122:3300/0 conn(0x7f202014d540 0x7f202014d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 --2- 172.21.15.71:0/4263456840 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f20200c3600 0x7f2020153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 -- 172.21.15.71:0/4263456840 >> 172.21.15.71:0/4263456840 conn(0x7f202006e220 msgr2=0x7f20200c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:35.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 -- 172.21.15.71:0/4263456840 shutdown_connections 2024-04-16T15:41:35.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.209+0000 7f2028655640 1 -- 172.21.15.71:0/4263456840 wait complete. 2024-04-16T15:41:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.210+0000 7f2028655640 1 Processor -- start 2024-04-16T15:41:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.210+0000 7f2028655640 1 -- start start 2024-04-16T15:41:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.210+0000 7f2028655640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020072f10 0x7f20201ea070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.210+0000 7f2028655640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200c3600 0x7f20200c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.210+0000 7f2028655640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f202014d540 0x7f2020151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2028655640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f202006d1f0 con 0x7f20200c3600 2024-04-16T15:41:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2028655640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f202006c420 con 0x7f2020072f10 2024-04-16T15:41:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2028655640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f20200c1d10 con 0x7f202014d540 2024-04-16T15:41:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200c3600 0x7f20200c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200c3600 0x7f20200c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53956/0 (socket says 172.21.15.71:53956) 2024-04-16T15:41:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 -- 172.21.15.71:0/350325608 learned_addr learned my addr 172.21.15.71:0/350325608 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2026bf2640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f202014d540 0x7f2020151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f20263f1640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020072f10 0x7f20201ea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 -- 172.21.15.71:0/350325608 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f202014d540 msgr2=0x7f2020151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f202014d540 0x7f2020151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 -- 172.21.15.71:0/350325608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020072f10 msgr2=0x7f20201ea070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020072f10 0x7f20201ea070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 -- 172.21.15.71:0/350325608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20200c6b00 con 0x7f20200c3600 2024-04-16T15:41:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2025bf0640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200c3600 0x7f20200c5c80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f201000d800 tx=0x7f201000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2016ffd640 1 -- 172.21.15.71:0/350325608 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201000b6c0 con 0x7f20200c3600 2024-04-16T15:41:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2016ffd640 1 -- 172.21.15.71:0/350325608 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2010019070 con 0x7f20200c3600 2024-04-16T15:41:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2020152d30 con 0x7f20200c3600 2024-04-16T15:41:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2016ffd640 1 -- 172.21.15.71:0/350325608 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201000ba80 con 0x7f20200c3600 2024-04-16T15:41:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.211+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20201532f0 con 0x7f20200c3600 2024-04-16T15:41:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.212+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2020151890 con 0x7f20200c3600 2024-04-16T15:41:35.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.213+0000 7f2016ffd640 1 -- 172.21.15.71:0/350325608 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f201000bc20 con 0x7f20200c3600 2024-04-16T15:41:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.213+0000 7f2016ffd640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f200406c8f0 0x7f200406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.213+0000 7f20263f1640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f200406c8f0 0x7f200406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.213+0000 7f2016ffd640 1 -- 172.21.15.71:0/350325608 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f201008bf90 con 0x7f20200c3600 2024-04-16T15:41:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.214+0000 7f20263f1640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f200406c8f0 0x7f200406edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20200c72a0 tx=0x7f201c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.217+0000 7f2016ffd640 1 -- 172.21.15.71:0/350325608 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20200c6b00 con 0x7f20200c3600 2024-04-16T15:41:35.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.396+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f20200c7540 con 0x7f200406c8f0 2024-04-16T15:41:35.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.396+0000 7f2016ffd640 1 -- 172.21.15.71:0/350325608 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f20200c7540 con 0x7f200406c8f0 2024-04-16T15:41:35.396 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 7 to deep-scrub 2024-04-16T15:41:35.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f200406c8f0 msgr2=0x7f200406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f200406c8f0 0x7f200406edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20200c72a0 tx=0x7f201c008040 comp rx=0 tx=0).stop 2024-04-16T15:41:35.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200c3600 msgr2=0x7f20200c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200c3600 0x7f20200c5c80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f201000d800 tx=0x7f201000dcc0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 shutdown_connections 2024-04-16T15:41:35.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f200406c8f0 0x7f200406edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f202014d540 0x7f2020151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200c3600 0x7f20200c5c80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 --2- 172.21.15.71:0/350325608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020072f10 0x7f20201ea070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 >> 172.21.15.71:0/350325608 conn(0x7f202006e220 msgr2=0x7f2020145cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:35.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 shutdown_connections 2024-04-16T15:41:35.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.398+0000 7f2028655640 1 -- 172.21.15.71:0/350325608 wait complete. 2024-04-16T15:41:35.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:41:35.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.512+0000 7f136f1f4640 1 Processor -- start 2024-04-16T15:41:35.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.512+0000 7f136f1f4640 1 -- start start 2024-04-16T15:41:35.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.513+0000 7f136f1f4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368153210 0x7f13680aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.513+0000 7f136f1f4640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1368156620 0x7f136806da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.513+0000 7f136f1f4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13680ab030 con 0x7f136806df90 2024-04-16T15:41:35.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.513+0000 7f136f1f4640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f13681535f0 con 0x7f1368156620 2024-04-16T15:41:35.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.513+0000 7f136f1f4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f13680c1c10 con 0x7f1368153210 2024-04-16T15:41:35.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.513+0000 7f135ffff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1368156620 0x7f136806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.513+0000 7f135ffff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f1368156620 0x7f136806da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53966/0 (socket says 172.21.15.71:53966) 2024-04-16T15:41:35.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.513+0000 7f135ffff640 1 -- 172.21.15.71:0/3685744539 learned_addr learned my addr 172.21.15.71:0/3685744539 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:35.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.514+0000 7f135ffff640 1 -- 172.21.15.71:0/3685744539 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368153210 msgr2=0x7f13680aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:35.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.514+0000 7f135ffff640 1 --2- 172.21.15.71:0/3685744539 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368153210 0x7f13680aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.514+0000 7f135ffff640 1 -- 172.21.15.71:0/3685744539 >> v1:172.21.15.122:6789/0 conn(0x7f136806df90 legacy=0x7f136806fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:35.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.514+0000 7f135ffff640 1 -- 172.21.15.71:0/3685744539 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13680c1ef0 con 0x7f1368156620 2024-04-16T15:41:35.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.514+0000 7f135ffff640 1 --2- 172.21.15.71:0/3685744539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368156620 0x7f136806da50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1350009870 tx=0x7f135002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b97068541ff97a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f1367fff640 1 -- 172.21.15.71:0/3685744539 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350038070 con 0x7f1368156620 2024-04-16T15:41:35.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f1367fff640 1 -- 172.21.15.71:0/3685744539 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1350031740 con 0x7f1368156620 2024-04-16T15:41:35.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f1367fff640 1 -- 172.21.15.71:0/3685744539 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135002f050 con 0x7f1368156620 2024-04-16T15:41:35.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f136f1f4640 1 -- 172.21.15.71:0/3685744539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368156620 msgr2=0x7f136806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3685744539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368156620 0x7f136806da50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1350009870 tx=0x7f135002b7e0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f136f1f4640 1 -- 172.21.15.71:0/3685744539 shutdown_connections 2024-04-16T15:41:35.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3685744539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368156620 0x7f136806da50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3685744539 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368153210 0x7f13680aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f136f1f4640 1 -- 172.21.15.71:0/3685744539 >> 172.21.15.71:0/3685744539 conn(0x7f1368072aa0 msgr2=0x7f1368072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:35.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f136f1f4640 1 -- 172.21.15.71:0/3685744539 shutdown_connections 2024-04-16T15:41:35.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.515+0000 7f136f1f4640 1 -- 172.21.15.71:0/3685744539 wait complete. 2024-04-16T15:41:35.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.516+0000 7f136f1f4640 1 Processor -- start 2024-04-16T15:41:35.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.516+0000 7f136f1f4640 1 -- start start 2024-04-16T15:41:35.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f136f1f4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f136806df90 0x7f13680c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f136f1f4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368153210 0x7f136814e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f136f1f4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368156620 0x7f13681e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f136f1f4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f13680712f0 con 0x7f1368153210 2024-04-16T15:41:35.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f136f1f4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1368156a80 con 0x7f136806df90 2024-04-16T15:41:35.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f136f1f4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1368152a80 con 0x7f1368156620 2024-04-16T15:41:35.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f135ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368153210 0x7f136814e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f135ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368153210 0x7f136814e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53980/0 (socket says 172.21.15.71:53980) 2024-04-16T15:41:35.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f135ffff640 1 -- 172.21.15.71:0/3549496798 learned_addr learned my addr 172.21.15.71:0/3549496798 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:35.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f136d791640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368156620 0x7f13681e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.517+0000 7f136cf90640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f136806df90 0x7f13680c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f135ffff640 1 -- 172.21.15.71:0/3549496798 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368156620 msgr2=0x7f13681e16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f135ffff640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368156620 0x7f13681e16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f135ffff640 1 -- 172.21.15.71:0/3549496798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f136806df90 msgr2=0x7f13680c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f135ffff640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f136806df90 0x7f13680c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f135ffff640 1 -- 172.21.15.71:0/3549496798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13681eba20 con 0x7f1368153210 2024-04-16T15:41:35.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f135ffff640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368153210 0x7f136814e1e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f13500090b0 tx=0x7f13500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f1365ffb640 1 -- 172.21.15.71:0/3549496798 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350043070 con 0x7f1368153210 2024-04-16T15:41:35.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f1365ffb640 1 -- 172.21.15.71:0/3549496798 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1350031ea0 con 0x7f1368153210 2024-04-16T15:41:35.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13681ebca0 con 0x7f1368153210 2024-04-16T15:41:35.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f1365ffb640 1 -- 172.21.15.71:0/3549496798 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350038040 con 0x7f1368153210 2024-04-16T15:41:35.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.518+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681ec260 con 0x7f1368153210 2024-04-16T15:41:35.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.519+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1368153800 con 0x7f1368153210 2024-04-16T15:41:35.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.519+0000 7f1365ffb640 1 -- 172.21.15.71:0/3549496798 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1350046070 con 0x7f1368153210 2024-04-16T15:41:35.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.520+0000 7f1365ffb640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f134806c8f0 0x7f134806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:35.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.520+0000 7f1365ffb640 1 -- 172.21.15.71:0/3549496798 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f13500ad8e0 con 0x7f1368153210 2024-04-16T15:41:35.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.520+0000 7f136cf90640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f134806c8f0 0x7f134806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:35.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.520+0000 7f136cf90640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f134806c8f0 0x7f134806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f135800a400 tx=0x7f135800d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:35.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.524+0000 7f1365ffb640 1 -- 172.21.15.71:0/3549496798 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1368153800 con 0x7f1368153210 2024-04-16T15:41:35.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.696+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13680c2620 con 0x7f134806c8f0 2024-04-16T15:41:35.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.701+0000 7f1365ffb640 1 -- 172.21.15.71:0/3549496798 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251847 (secure 0 0 0) 0x7f13680c2620 con 0x7f134806c8f0 2024-04-16T15:41:35.704 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:35.705 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:41:35.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f134806c8f0 msgr2=0x7f134806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f134806c8f0 0x7f134806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f135800a400 tx=0x7f135800d040 comp rx=0 tx=0).stop 2024-04-16T15:41:35.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368153210 msgr2=0x7f136814e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:35.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368153210 0x7f136814e1e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f13500090b0 tx=0x7f13500046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 shutdown_connections 2024-04-16T15:41:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f134806c8f0 0x7f134806edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1368156620 0x7f13681e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1368153210 0x7f136814e1e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 --2- 172.21.15.71:0/3549496798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f136806df90 0x7f13680c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 >> 172.21.15.71:0/3549496798 conn(0x7f1368072aa0 msgr2=0x7f136806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 shutdown_connections 2024-04-16T15:41:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:35.705+0000 7f136f1f4640 1 -- 172.21.15.71:0/3549496798 wait complete. 2024-04-16T15:41:35.716 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-16T15:41:33.714258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220816,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753904,"statfs":{"total":773094113280,"available":772867997696,"internally_reserved":0,"allocated":4759552,"data_stored":2707140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.006997"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530182+0000","last_change":"2024-04-16T15:41:23.523079+0000","last_active":"2024-04-16T15:41:24.530182+0000","last_peered":"2024-04-16T15:41:24.530182+0000","last_clean":"2024-04-16T15:41:24.530182+0000","last_became_active":"2024-04-16T15:41:23.522804+0000","last_became_peered":"2024-04-16T15:41:23.522804+0000","last_unstale":"2024-04-16T15:41:24.530182+0000","last_undegraded":"2024-04-16T15:41:24.530182+0000","last_fullsized":"2024-04-16T15:41:24.530182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530026+0000","last_change":"2024-04-16T15:41:23.518347+0000","last_active":"2024-04-16T15:41:24.530026+0000","last_peered":"2024-04-16T15:41:24.530026+0000","last_clean":"2024-04-16T15:41:24.530026+0000","last_became_active":"2024-04-16T15:41:23.517372+0000","last_became_peered":"2024-04-16T15:41:23.517372+0000","last_unstale":"2024-04-16T15:41:24.530026+0000","last_undegraded":"2024-04-16T15:41:24.530026+0000","last_fullsized":"2024-04-16T15:41:24.530026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.533697+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.533520+0000","last_became_peered":"2024-04-16T15:41:23.533520+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069345+0000","last_change":"2024-04-16T15:41:23.521655+0000","last_active":"2024-04-16T15:41:25.069345+0000","last_peered":"2024-04-16T15:41:25.069345+0000","last_clean":"2024-04-16T15:41:25.069345+0000","last_became_active":"2024-04-16T15:41:23.521419+0000","last_became_peered":"2024-04-16T15:41:23.521419+0000","last_unstale":"2024-04-16T15:41:25.069345+0000","last_undegraded":"2024-04-16T15:41:25.069345+0000","last_fullsized":"2024-04-16T15:41:25.069345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:48.607343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069423+0000","last_change":"2024-04-16T15:41:23.522246+0000","last_active":"2024-04-16T15:41:25.069423+0000","last_peered":"2024-04-16T15:41:25.069423+0000","last_clean":"2024-04-16T15:41:25.069423+0000","last_became_active":"2024-04-16T15:41:23.522009+0000","last_became_peered":"2024-04-16T15:41:23.522009+0000","last_unstale":"2024-04-16T15:41:25.069423+0000","last_undegraded":"2024-04-16T15:41:25.069423+0000","last_fullsized":"2024-04-16T15:41:25.069423+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:33.728092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069091+0000","last_change":"2024-04-16T15:41:23.523004+0000","last_active":"2024-04-16T15:41:25.069091+0000","last_peered":"2024-04-16T15:41:25.069091+0000","last_clean":"2024-04-16T15:41:25.069091+0000","last_became_active":"2024-04-16T15:41:23.522742+0000","last_became_peered":"2024-04-16T15:41:23.522742+0000","last_unstale":"2024-04-16T15:41:25.069091+0000","last_undegraded":"2024-04-16T15:41:25.069091+0000","last_fullsized":"2024-04-16T15:41:25.069091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527561+0000","last_change":"2024-04-16T15:41:23.523305+0000","last_active":"2024-04-16T15:41:24.527561+0000","last_peered":"2024-04-16T15:41:24.527561+0000","last_clean":"2024-04-16T15:41:24.527561+0000","last_became_active":"2024-04-16T15:41:23.523022+0000","last_became_peered":"2024-04-16T15:41:23.523022+0000","last_unstale":"2024-04-16T15:41:24.527561+0000","last_undegraded":"2024-04-16T15:41:24.527561+0000","last_fullsized":"2024-04-16T15:41:24.527561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529666+0000","last_change":"2024-04-16T15:41:23.535446+0000","last_active":"2024-04-16T15:41:24.529666+0000","last_peered":"2024-04-16T15:41:24.529666+0000","last_clean":"2024-04-16T15:41:24.529666+0000","last_became_active":"2024-04-16T15:41:23.535156+0000","last_became_peered":"2024-04-16T15:41:23.535156+0000","last_unstale":"2024-04-16T15:41:24.529666+0000","last_undegraded":"2024-04-16T15:41:24.529666+0000","last_fullsized":"2024-04-16T15:41:24.529666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069784+0000","last_change":"2024-04-16T15:41:23.535019+0000","last_active":"2024-04-16T15:41:25.069784+0000","last_peered":"2024-04-16T15:41:25.069784+0000","last_clean":"2024-04-16T15:41:25.069784+0000","last_became_active":"2024-04-16T15:41:23.534726+0000","last_became_peered":"2024-04-16T15:41:23.534726+0000","last_unstale":"2024-04-16T15:41:25.069784+0000","last_undegraded":"2024-04-16T15:41:25.069784+0000","last_fullsized":"2024-04-16T15:41:25.069784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528288+0000","last_change":"2024-04-16T15:41:23.522709+0000","last_active":"2024-04-16T15:41:24.528288+0000","last_peered":"2024-04-16T15:41:24.528288+0000","last_clean":"2024-04-16T15:41:24.528288+0000","last_became_active":"2024-04-16T15:41:23.522437+0000","last_became_peered":"2024-04-16T15:41:23.522437+0000","last_unstale":"2024-04-16T15:41:24.528288+0000","last_undegraded":"2024-04-16T15:41:24.528288+0000","last_fullsized":"2024-04-16T15:41:24.528288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528764+0000","last_change":"2024-04-16T15:41:23.530620+0000","last_active":"2024-04-16T15:41:24.528764+0000","last_peered":"2024-04-16T15:41:24.528764+0000","last_clean":"2024-04-16T15:41:24.528764+0000","last_became_active":"2024-04-16T15:41:23.530320+0000","last_became_peered":"2024-04-16T15:41:23.530320+0000","last_unstale":"2024-04-16T15:41:24.528764+0000","last_undegraded":"2024-04-16T15:41:24.528764+0000","last_fullsized":"2024-04-16T15:41:24.528764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:34.078895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068927+0000","last_change":"2024-04-16T15:41:23.523419+0000","last_active":"2024-04-16T15:41:25.068927+0000","last_peered":"2024-04-16T15:41:25.068927+0000","last_clean":"2024-04-16T15:41:25.068927+0000","last_became_active":"2024-04-16T15:41:23.523081+0000","last_became_peered":"2024-04-16T15:41:23.523081+0000","last_unstale":"2024-04-16T15:41:25.068927+0000","last_undegraded":"2024-04-16T15:41:25.068927+0000","last_fullsized":"2024-04-16T15:41:25.068927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527828+0000","last_change":"2024-04-16T15:41:23.522227+0000","last_active":"2024-04-16T15:41:24.527828+0000","last_peered":"2024-04-16T15:41:24.527828+0000","last_clean":"2024-04-16T15:41:24.527828+0000","last_became_active":"2024-04-16T15:41:23.521531+0000","last_became_peered":"2024-04-16T15:41:23.521531+0000","last_unstale":"2024-04-16T15:41:24.527828+0000","last_undegraded":"2024-04-16T15:41:24.527828+0000","last_fullsized":"2024-04-16T15:41:24.527828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683427+0000","last_change":"2024-04-16T15:41:23.529710+0000","last_active":"2024-04-16T15:41:24.683427+0000","last_peered":"2024-04-16T15:41:24.683427+0000","last_clean":"2024-04-16T15:41:24.683427+0000","last_became_active":"2024-04-16T15:41:23.528557+0000","last_became_peered":"2024-04-16T15:41:23.528557+0000","last_unstale":"2024-04-16T15:41:24.683427+0000","last_undegraded":"2024-04-16T15:41:24.683427+0000","last_fullsized":"2024-04-16T15:41:24.683427+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527689+0000","last_change":"2024-04-16T15:41:23.513641+0000","last_active":"2024-04-16T15:41:24.527689+0000","last_peered":"2024-04-16T15:41:24.527689+0000","last_clean":"2024-04-16T15:41:24.527689+0000","last_became_active":"2024-04-16T15:41:23.513340+0000","last_became_peered":"2024-04-16T15:41:23.513340+0000","last_unstale":"2024-04-16T15:41:24.527689+0000","last_undegraded":"2024-04-16T15:41:24.527689+0000","last_fullsized":"2024-04-16T15:41:24.527689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527345+0000","last_change":"2024-04-16T15:41:23.535782+0000","last_active":"2024-04-16T15:41:24.527345+0000","last_peered":"2024-04-16T15:41:24.527345+0000","last_clean":"2024-04-16T15:41:24.527345+0000","last_became_active":"2024-04-16T15:41:23.535549+0000","last_became_peered":"2024-04-16T15:41:23.535549+0000","last_unstale":"2024-04-16T15:41:24.527345+0000","last_undegraded":"2024-04-16T15:41:24.527345+0000","last_fullsized":"2024-04-16T15:41:24.527345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528422+0000","last_change":"2024-04-16T15:41:23.536402+0000","last_active":"2024-04-16T15:41:24.528422+0000","last_peered":"2024-04-16T15:41:24.528422+0000","last_clean":"2024-04-16T15:41:24.528422+0000","last_became_active":"2024-04-16T15:41:23.536009+0000","last_became_peered":"2024-04-16T15:41:23.536009+0000","last_unstale":"2024-04-16T15:41:24.528422+0000","last_undegraded":"2024-04-16T15:41:24.528422+0000","last_fullsized":"2024-04-16T15:41:24.528422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068843+0000","last_change":"2024-04-16T15:41:23.523701+0000","last_active":"2024-04-16T15:41:25.068843+0000","last_peered":"2024-04-16T15:41:25.068843+0000","last_clean":"2024-04-16T15:41:25.068843+0000","last_became_active":"2024-04-16T15:41:23.523442+0000","last_became_peered":"2024-04-16T15:41:23.523442+0000","last_unstale":"2024-04-16T15:41:25.068843+0000","last_undegraded":"2024-04-16T15:41:25.068843+0000","last_fullsized":"2024-04-16T15:41:25.068843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070293+0000","last_change":"2024-04-16T15:41:23.519553+0000","last_active":"2024-04-16T15:41:25.070293+0000","last_peered":"2024-04-16T15:41:25.070293+0000","last_clean":"2024-04-16T15:41:25.070293+0000","last_became_active":"2024-04-16T15:41:23.519258+0000","last_became_peered":"2024-04-16T15:41:23.519258+0000","last_unstale":"2024-04-16T15:41:25.070293+0000","last_undegraded":"2024-04-16T15:41:25.070293+0000","last_fullsized":"2024-04-16T15:41:25.070293+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:51.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068696+0000","last_change":"2024-04-16T15:41:23.522212+0000","last_active":"2024-04-16T15:41:25.068696+0000","last_peered":"2024-04-16T15:41:25.068696+0000","last_clean":"2024-04-16T15:41:25.068696+0000","last_became_active":"2024-04-16T15:41:23.521934+0000","last_became_peered":"2024-04-16T15:41:23.521934+0000","last_unstale":"2024-04-16T15:41:25.068696+0000","last_undegraded":"2024-04-16T15:41:25.068696+0000","last_fullsized":"2024-04-16T15:41:25.068696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:03.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527973+0000","last_change":"2024-04-16T15:41:23.520977+0000","last_active":"2024-04-16T15:41:24.527973+0000","last_peered":"2024-04-16T15:41:24.527973+0000","last_clean":"2024-04-16T15:41:24.527973+0000","last_became_active":"2024-04-16T15:41:23.520677+0000","last_became_peered":"2024-04-16T15:41:23.520677+0000","last_unstale":"2024-04-16T15:41:24.527973+0000","last_undegraded":"2024-04-16T15:41:24.527973+0000","last_fullsized":"2024-04-16T15:41:24.527973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683729+0000","last_change":"2024-04-16T15:41:23.527968+0000","last_active":"2024-04-16T15:41:24.683729+0000","last_peered":"2024-04-16T15:41:24.683729+0000","last_clean":"2024-04-16T15:41:24.683729+0000","last_became_active":"2024-04-16T15:41:23.527690+0000","last_became_peered":"2024-04-16T15:41:23.527690+0000","last_unstale":"2024-04-16T15:41:24.683729+0000","last_undegraded":"2024-04-16T15:41:24.683729+0000","last_fullsized":"2024-04-16T15:41:24.683729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527416+0000","last_change":"2024-04-16T15:41:23.527691+0000","last_active":"2024-04-16T15:41:24.527416+0000","last_peered":"2024-04-16T15:41:24.527416+0000","last_clean":"2024-04-16T15:41:24.527416+0000","last_became_active":"2024-04-16T15:41:23.527372+0000","last_became_peered":"2024-04-16T15:41:23.527372+0000","last_unstale":"2024-04-16T15:41:24.527416+0000","last_undegraded":"2024-04-16T15:41:24.527416+0000","last_fullsized":"2024-04-16T15:41:24.527416+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069312+0000","last_change":"2024-04-16T15:41:23.525188+0000","last_active":"2024-04-16T15:41:25.069312+0000","last_peered":"2024-04-16T15:41:25.069312+0000","last_clean":"2024-04-16T15:41:25.069312+0000","last_became_active":"2024-04-16T15:41:23.524939+0000","last_became_peered":"2024-04-16T15:41:23.524939+0000","last_unstale":"2024-04-16T15:41:25.069312+0000","last_undegraded":"2024-04-16T15:41:25.069312+0000","last_fullsized":"2024-04-16T15:41:25.069312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:53.692430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068745+0000","last_change":"2024-04-16T15:41:23.526266+0000","last_active":"2024-04-16T15:41:25.068745+0000","last_peered":"2024-04-16T15:41:25.068745+0000","last_clean":"2024-04-16T15:41:25.068745+0000","last_became_active":"2024-04-16T15:41:23.525999+0000","last_became_peered":"2024-04-16T15:41:23.525999+0000","last_unstale":"2024-04-16T15:41:25.068745+0000","last_undegraded":"2024-04-16T15:41:25.068745+0000","last_fullsized":"2024-04-16T15:41:25.068745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:38.014259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069155+0000","last_change":"2024-04-16T15:41:23.535525+0000","last_active":"2024-04-16T15:41:25.069155+0000","last_peered":"2024-04-16T15:41:25.069155+0000","last_clean":"2024-04-16T15:41:25.069155+0000","last_became_active":"2024-04-16T15:41:23.535192+0000","last_became_peered":"2024-04-16T15:41:23.535192+0000","last_unstale":"2024-04-16T15:41:25.069155+0000","last_undegraded":"2024-04-16T15:41:25.069155+0000","last_fullsized":"2024-04-16T15:41:25.069155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068809+0000","last_change":"2024-04-16T15:41:23.527164+0000","last_active":"2024-04-16T15:41:25.068809+0000","last_peered":"2024-04-16T15:41:25.068809+0000","last_clean":"2024-04-16T15:41:25.068809+0000","last_became_active":"2024-04-16T15:41:23.526765+0000","last_became_peered":"2024-04-16T15:41:23.526765+0000","last_unstale":"2024-04-16T15:41:25.068809+0000","last_undegraded":"2024-04-16T15:41:25.068809+0000","last_fullsized":"2024-04-16T15:41:25.068809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:18.557462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528932+0000","last_change":"2024-04-16T15:41:23.529704+0000","last_active":"2024-04-16T15:41:24.528932+0000","last_peered":"2024-04-16T15:41:24.528932+0000","last_clean":"2024-04-16T15:41:24.528932+0000","last_became_active":"2024-04-16T15:41:23.525283+0000","last_became_peered":"2024-04-16T15:41:23.525283+0000","last_unstale":"2024-04-16T15:41:24.528932+0000","last_undegraded":"2024-04-16T15:41:24.528932+0000","last_fullsized":"2024-04-16T15:41:24.528932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068718+0000","last_change":"2024-04-16T15:41:23.523057+0000","last_active":"2024-04-16T15:41:25.068718+0000","last_peered":"2024-04-16T15:41:25.068718+0000","last_clean":"2024-04-16T15:41:25.068718+0000","last_became_active":"2024-04-16T15:41:23.522762+0000","last_became_peered":"2024-04-16T15:41:23.522762+0000","last_unstale":"2024-04-16T15:41:25.068718+0000","last_undegraded":"2024-04-16T15:41:25.068718+0000","last_fullsized":"2024-04-16T15:41:25.068718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:57.277773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068960+0000","last_change":"2024-04-16T15:41:23.522445+0000","last_active":"2024-04-16T15:41:25.068960+0000","last_peered":"2024-04-16T15:41:25.068960+0000","last_clean":"2024-04-16T15:41:25.068960+0000","last_became_active":"2024-04-16T15:41:23.522179+0000","last_became_peered":"2024-04-16T15:41:23.522179+0000","last_unstale":"2024-04-16T15:41:25.068960+0000","last_undegraded":"2024-04-16T15:41:25.068960+0000","last_fullsized":"2024-04-16T15:41:25.068960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527557+0000","last_change":"2024-04-16T15:41:23.528582+0000","last_active":"2024-04-16T15:41:24.527557+0000","last_peered":"2024-04-16T15:41:24.527557+0000","last_clean":"2024-04-16T15:41:24.527557+0000","last_became_active":"2024-04-16T15:41:23.528304+0000","last_became_peered":"2024-04-16T15:41:23.528304+0000","last_unstale":"2024-04-16T15:41:24.527557+0000","last_undegraded":"2024-04-16T15:41:24.527557+0000","last_fullsized":"2024-04-16T15:41:24.527557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:58:58.124408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069657+0000","last_change":"2024-04-16T15:41:23.519347+0000","last_active":"2024-04-16T15:41:25.069657+0000","last_peered":"2024-04-16T15:41:25.069657+0000","last_clean":"2024-04-16T15:41:25.069657+0000","last_became_active":"2024-04-16T15:41:23.519092+0000","last_became_peered":"2024-04-16T15:41:23.519092+0000","last_unstale":"2024-04-16T15:41:25.069657+0000","last_undegraded":"2024-04-16T15:41:25.069657+0000","last_fullsized":"2024-04-16T15:41:25.069657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527966+0000","last_change":"2024-04-16T15:41:23.536251+0000","last_active":"2024-04-16T15:41:24.527966+0000","last_peered":"2024-04-16T15:41:24.527966+0000","last_clean":"2024-04-16T15:41:24.527966+0000","last_became_active":"2024-04-16T15:41:23.535963+0000","last_became_peered":"2024-04-16T15:41:23.535963+0000","last_unstale":"2024-04-16T15:41:24.527966+0000","last_undegraded":"2024-04-16T15:41:24.527966+0000","last_fullsized":"2024-04-16T15:41:24.527966+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:16:57.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068769+0000","last_change":"2024-04-16T15:41:22.522796+0000","last_active":"2024-04-16T15:41:25.068769+0000","last_peered":"2024-04-16T15:41:25.068769+0000","last_clean":"2024-04-16T15:41:25.068769+0000","last_became_active":"2024-04-16T15:41:22.522542+0000","last_became_peered":"2024-04-16T15:41:22.522542+0000","last_unstale":"2024-04-16T15:41:25.068769+0000","last_undegraded":"2024-04-16T15:41:25.068769+0000","last_fullsized":"2024-04-16T15:41:25.068769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069565+0000","last_change":"2024-04-16T15:41:23.536192+0000","last_active":"2024-04-16T15:41:25.069565+0000","last_peered":"2024-04-16T15:41:25.069565+0000","last_clean":"2024-04-16T15:41:25.069565+0000","last_became_active":"2024-04-16T15:41:23.535920+0000","last_became_peered":"2024-04-16T15:41:23.535920+0000","last_unstale":"2024-04-16T15:41:25.069565+0000","last_undegraded":"2024-04-16T15:41:25.069565+0000","last_fullsized":"2024-04-16T15:41:25.069565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069482+0000","last_change":"2024-04-16T15:41:22.523464+0000","last_active":"2024-04-16T15:41:25.069482+0000","last_peered":"2024-04-16T15:41:25.069482+0000","last_clean":"2024-04-16T15:41:25.069482+0000","last_became_active":"2024-04-16T15:41:22.523128+0000","last_became_peered":"2024-04-16T15:41:22.523128+0000","last_unstale":"2024-04-16T15:41:25.069482+0000","last_undegraded":"2024-04-16T15:41:25.069482+0000","last_fullsized":"2024-04-16T15:41:25.069482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683674+0000","last_change":"2024-04-16T15:41:23.530373+0000","last_active":"2024-04-16T15:41:24.683674+0000","last_peered":"2024-04-16T15:41:24.683674+0000","last_clean":"2024-04-16T15:41:24.683674+0000","last_became_active":"2024-04-16T15:41:23.530115+0000","last_became_peered":"2024-04-16T15:41:23.530115+0000","last_unstale":"2024-04-16T15:41:24.683674+0000","last_undegraded":"2024-04-16T15:41:24.683674+0000","last_fullsized":"2024-04-16T15:41:24.683674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:18:05.663505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069538+0000","last_change":"2024-04-16T15:41:22.521948+0000","last_active":"2024-04-16T15:41:25.069538+0000","last_peered":"2024-04-16T15:41:25.069538+0000","last_clean":"2024-04-16T15:41:25.069538+0000","last_became_active":"2024-04-16T15:41:22.521643+0000","last_became_peered":"2024-04-16T15:41:22.521643+0000","last_unstale":"2024-04-16T15:41:25.069538+0000","last_undegraded":"2024-04-16T15:41:25.069538+0000","last_fullsized":"2024-04-16T15:41:25.069538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530028+0000","last_change":"2024-04-16T15:41:23.528874+0000","last_active":"2024-04-16T15:41:24.530028+0000","last_peered":"2024-04-16T15:41:24.530028+0000","last_clean":"2024-04-16T15:41:24.530028+0000","last_became_active":"2024-04-16T15:41:23.528569+0000","last_became_peered":"2024-04-16T15:41:23.528569+0000","last_unstale":"2024-04-16T15:41:24.530028+0000","last_undegraded":"2024-04-16T15:41:24.530028+0000","last_fullsized":"2024-04-16T15:41:24.530028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683536+0000","last_change":"2024-04-16T15:41:22.525207+0000","last_active":"2024-04-16T15:41:24.683536+0000","last_peered":"2024-04-16T15:41:24.683536+0000","last_clean":"2024-04-16T15:41:24.683536+0000","last_became_active":"2024-04-16T15:41:22.524864+0000","last_became_peered":"2024-04-16T15:41:22.524864+0000","last_unstale":"2024-04-16T15:41:24.683536+0000","last_undegraded":"2024-04-16T15:41:24.683536+0000","last_fullsized":"2024-04-16T15:41:24.683536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068873+0000","last_change":"2024-04-16T15:41:23.523861+0000","last_active":"2024-04-16T15:41:25.068873+0000","last_peered":"2024-04-16T15:41:25.068873+0000","last_clean":"2024-04-16T15:41:25.068873+0000","last_became_active":"2024-04-16T15:41:23.523587+0000","last_became_peered":"2024-04-16T15:41:23.523587+0000","last_unstale":"2024-04-16T15:41:25.068873+0000","last_undegraded":"2024-04-16T15:41:25.068873+0000","last_fullsized":"2024-04-16T15:41:25.068873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:22.526265+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:22.525849+0000","last_became_peered":"2024-04-16T15:41:22.525849+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:47:02.400517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528806+0000","last_change":"2024-04-16T15:41:23.521523+0000","last_active":"2024-04-16T15:41:24.528806+0000","last_peered":"2024-04-16T15:41:24.528806+0000","last_clean":"2024-04-16T15:41:24.528806+0000","last_became_active":"2024-04-16T15:41:23.520768+0000","last_became_peered":"2024-04-16T15:41:23.520768+0000","last_unstale":"2024-04-16T15:41:24.528806+0000","last_undegraded":"2024-04-16T15:41:24.528806+0000","last_fullsized":"2024-04-16T15:41:24.528806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527590+0000","last_change":"2024-04-16T15:41:22.528402+0000","last_active":"2024-04-16T15:41:24.527590+0000","last_peered":"2024-04-16T15:41:24.527590+0000","last_clean":"2024-04-16T15:41:24.527590+0000","last_became_active":"2024-04-16T15:41:22.527999+0000","last_became_peered":"2024-04-16T15:41:22.527999+0000","last_unstale":"2024-04-16T15:41:24.527590+0000","last_undegraded":"2024-04-16T15:41:24.527590+0000","last_fullsized":"2024-04-16T15:41:24.527590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683644+0000","last_change":"2024-04-16T15:41:23.535673+0000","last_active":"2024-04-16T15:41:24.683644+0000","last_peered":"2024-04-16T15:41:24.683644+0000","last_clean":"2024-04-16T15:41:24.683644+0000","last_became_active":"2024-04-16T15:41:23.535387+0000","last_became_peered":"2024-04-16T15:41:23.535387+0000","last_unstale":"2024-04-16T15:41:24.683644+0000","last_undegraded":"2024-04-16T15:41:24.683644+0000","last_fullsized":"2024-04-16T15:41:24.683644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:42.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070111+0000","last_change":"2024-04-16T15:41:22.526184+0000","last_active":"2024-04-16T15:41:25.070111+0000","last_peered":"2024-04-16T15:41:25.070111+0000","last_clean":"2024-04-16T15:41:25.070111+0000","last_became_active":"2024-04-16T15:41:22.525819+0000","last_became_peered":"2024-04-16T15:41:22.525819+0000","last_unstale":"2024-04-16T15:41:25.070111+0000","last_undegraded":"2024-04-16T15:41:25.070111+0000","last_fullsized":"2024-04-16T15:41:25.070111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530001+0000","last_change":"2024-04-16T15:41:23.522193+0000","last_active":"2024-04-16T15:41:24.530001+0000","last_peered":"2024-04-16T15:41:24.530001+0000","last_clean":"2024-04-16T15:41:24.530001+0000","last_became_active":"2024-04-16T15:41:23.521900+0000","last_became_peered":"2024-04-16T15:41:23.521900+0000","last_unstale":"2024-04-16T15:41:24.530001+0000","last_undegraded":"2024-04-16T15:41:24.530001+0000","last_fullsized":"2024-04-16T15:41:24.530001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:19.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.521519+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.521234+0000","last_became_peered":"2024-04-16T15:41:22.521234+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:28:52.686958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069105+0000","last_change":"2024-04-16T15:41:23.521070+0000","last_active":"2024-04-16T15:41:25.069105+0000","last_peered":"2024-04-16T15:41:25.069105+0000","last_clean":"2024-04-16T15:41:25.069105+0000","last_became_active":"2024-04-16T15:41:23.520819+0000","last_became_peered":"2024-04-16T15:41:23.520819+0000","last_unstale":"2024-04-16T15:41:25.069105+0000","last_undegraded":"2024-04-16T15:41:25.069105+0000","last_fullsized":"2024-04-16T15:41:25.069105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:03:41.341706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530092+0000","last_change":"2024-04-16T15:41:22.519163+0000","last_active":"2024-04-16T15:41:24.530092+0000","last_peered":"2024-04-16T15:41:24.530092+0000","last_clean":"2024-04-16T15:41:24.530092+0000","last_became_active":"2024-04-16T15:41:22.518905+0000","last_became_peered":"2024-04-16T15:41:22.518905+0000","last_unstale":"2024-04-16T15:41:24.530092+0000","last_undegraded":"2024-04-16T15:41:24.530092+0000","last_fullsized":"2024-04-16T15:41:24.530092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:03.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528974+0000","last_change":"2024-04-16T15:41:23.528995+0000","last_active":"2024-04-16T15:41:24.528974+0000","last_peered":"2024-04-16T15:41:24.528974+0000","last_clean":"2024-04-16T15:41:24.528974+0000","last_became_active":"2024-04-16T15:41:23.526288+0000","last_became_peered":"2024-04-16T15:41:23.526288+0000","last_unstale":"2024-04-16T15:41:24.528974+0000","last_undegraded":"2024-04-16T15:41:24.528974+0000","last_fullsized":"2024-04-16T15:41:24.528974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528459+0000","last_change":"2024-04-16T15:41:22.525433+0000","last_active":"2024-04-16T15:41:24.528459+0000","last_peered":"2024-04-16T15:41:24.528459+0000","last_clean":"2024-04-16T15:41:24.528459+0000","last_became_active":"2024-04-16T15:41:22.524567+0000","last_became_peered":"2024-04-16T15:41:22.524567+0000","last_unstale":"2024-04-16T15:41:24.528459+0000","last_undegraded":"2024-04-16T15:41:24.528459+0000","last_fullsized":"2024-04-16T15:41:24.528459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:20:56.272300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069430+0000","last_change":"2024-04-16T15:41:23.522367+0000","last_active":"2024-04-16T15:41:25.069430+0000","last_peered":"2024-04-16T15:41:25.069430+0000","last_clean":"2024-04-16T15:41:25.069430+0000","last_became_active":"2024-04-16T15:41:23.521984+0000","last_became_peered":"2024-04-16T15:41:23.521984+0000","last_unstale":"2024-04-16T15:41:25.069430+0000","last_undegraded":"2024-04-16T15:41:25.069430+0000","last_fullsized":"2024-04-16T15:41:25.069430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:09.002854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528812+0000","last_change":"2024-04-16T15:41:22.523086+0000","last_active":"2024-04-16T15:41:24.528812+0000","last_peered":"2024-04-16T15:41:24.528812+0000","last_clean":"2024-04-16T15:41:24.528812+0000","last_became_active":"2024-04-16T15:41:22.522759+0000","last_became_peered":"2024-04-16T15:41:22.522759+0000","last_unstale":"2024-04-16T15:41:24.528812+0000","last_undegraded":"2024-04-16T15:41:24.528812+0000","last_fullsized":"2024-04-16T15:41:24.528812+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683778+0000","last_change":"2024-04-16T15:41:23.529819+0000","last_active":"2024-04-16T15:41:24.683778+0000","last_peered":"2024-04-16T15:41:24.683778+0000","last_clean":"2024-04-16T15:41:24.683778+0000","last_became_active":"2024-04-16T15:41:23.529543+0000","last_became_peered":"2024-04-16T15:41:23.529543+0000","last_unstale":"2024-04-16T15:41:24.683778+0000","last_undegraded":"2024-04-16T15:41:24.683778+0000","last_fullsized":"2024-04-16T15:41:24.683778+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:38:18.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:22.522411+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:22.522130+0000","last_became_peered":"2024-04-16T15:41:22.522130+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:27:41.323112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069849+0000","last_change":"2024-04-16T15:41:23.535122+0000","last_active":"2024-04-16T15:41:25.069849+0000","last_peered":"2024-04-16T15:41:25.069849+0000","last_clean":"2024-04-16T15:41:25.069849+0000","last_became_active":"2024-04-16T15:41:23.534825+0000","last_became_peered":"2024-04-16T15:41:23.534825+0000","last_unstale":"2024-04-16T15:41:25.069849+0000","last_undegraded":"2024-04-16T15:41:25.069849+0000","last_fullsized":"2024-04-16T15:41:25.069849+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:21:33.160692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530105+0000","last_change":"2024-04-16T15:41:22.520252+0000","last_active":"2024-04-16T15:41:24.530105+0000","last_peered":"2024-04-16T15:41:24.530105+0000","last_clean":"2024-04-16T15:41:24.530105+0000","last_became_active":"2024-04-16T15:41:22.519829+0000","last_became_peered":"2024-04-16T15:41:22.519829+0000","last_unstale":"2024-04-16T15:41:24.530105+0000","last_undegraded":"2024-04-16T15:41:24.530105+0000","last_fullsized":"2024-04-16T15:41:24.530105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529732+0000","last_change":"2024-04-16T15:41:23.522432+0000","last_active":"2024-04-16T15:41:24.529732+0000","last_peered":"2024-04-16T15:41:24.529732+0000","last_clean":"2024-04-16T15:41:24.529732+0000","last_became_active":"2024-04-16T15:41:23.522159+0000","last_became_peered":"2024-04-16T15:41:23.522159+0000","last_unstale":"2024-04-16T15:41:24.529732+0000","last_undegraded":"2024-04-16T15:41:24.529732+0000","last_fullsized":"2024-04-16T15:41:24.529732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:20:47.888656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069145+0000","last_change":"2024-04-16T15:41:22.527064+0000","last_active":"2024-04-16T15:41:25.069145+0000","last_peered":"2024-04-16T15:41:25.069145+0000","last_clean":"2024-04-16T15:41:25.069145+0000","last_became_active":"2024-04-16T15:41:22.525811+0000","last_became_peered":"2024-04-16T15:41:22.525811+0000","last_unstale":"2024-04-16T15:41:25.069145+0000","last_undegraded":"2024-04-16T15:41:25.069145+0000","last_fullsized":"2024-04-16T15:41:25.069145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527532+0000","last_change":"2024-04-16T15:41:23.527859+0000","last_active":"2024-04-16T15:41:24.527532+0000","last_peered":"2024-04-16T15:41:24.527532+0000","last_clean":"2024-04-16T15:41:24.527532+0000","last_became_active":"2024-04-16T15:41:23.527295+0000","last_became_peered":"2024-04-16T15:41:23.527295+0000","last_unstale":"2024-04-16T15:41:24.527532+0000","last_undegraded":"2024-04-16T15:41:24.527532+0000","last_fullsized":"2024-04-16T15:41:24.527532+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:00:18.934742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068514+0000","last_change":"2024-04-16T15:41:22.524475+0000","last_active":"2024-04-16T15:41:25.068514+0000","last_peered":"2024-04-16T15:41:25.068514+0000","last_clean":"2024-04-16T15:41:25.068514+0000","last_became_active":"2024-04-16T15:41:22.523303+0000","last_became_peered":"2024-04-16T15:41:22.523303+0000","last_unstale":"2024-04-16T15:41:25.068514+0000","last_undegraded":"2024-04-16T15:41:25.068514+0000","last_fullsized":"2024-04-16T15:41:25.068514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529868+0000","last_change":"2024-04-16T15:41:23.523183+0000","last_active":"2024-04-16T15:41:24.529868+0000","last_peered":"2024-04-16T15:41:24.529868+0000","last_clean":"2024-04-16T15:41:24.529868+0000","last_became_active":"2024-04-16T15:41:23.522845+0000","last_became_peered":"2024-04-16T15:41:23.522845+0000","last_unstale":"2024-04-16T15:41:24.529868+0000","last_undegraded":"2024-04-16T15:41:24.529868+0000","last_fullsized":"2024-04-16T15:41:24.529868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:15.236135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527394+0000","last_change":"2024-04-16T15:41:22.522941+0000","last_active":"2024-04-16T15:41:24.527394+0000","last_peered":"2024-04-16T15:41:24.527394+0000","last_clean":"2024-04-16T15:41:24.527394+0000","last_became_active":"2024-04-16T15:41:22.522633+0000","last_became_peered":"2024-04-16T15:41:22.522633+0000","last_unstale":"2024-04-16T15:41:24.527394+0000","last_undegraded":"2024-04-16T15:41:24.527394+0000","last_fullsized":"2024-04-16T15:41:24.527394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:25:23.286296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529991+0000","last_change":"2024-04-16T15:41:23.528140+0000","last_active":"2024-04-16T15:41:24.529991+0000","last_peered":"2024-04-16T15:41:24.529991+0000","last_clean":"2024-04-16T15:41:24.529991+0000","last_became_active":"2024-04-16T15:41:23.527841+0000","last_became_peered":"2024-04-16T15:41:23.527841+0000","last_unstale":"2024-04-16T15:41:24.529991+0000","last_undegraded":"2024-04-16T15:41:24.529991+0000","last_fullsized":"2024-04-16T15:41:24.529991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528233+0000","last_change":"2024-04-16T15:41:22.526961+0000","last_active":"2024-04-16T15:41:24.528233+0000","last_peered":"2024-04-16T15:41:24.528233+0000","last_clean":"2024-04-16T15:41:24.528233+0000","last_became_active":"2024-04-16T15:41:22.526140+0000","last_became_peered":"2024-04-16T15:41:22.526140+0000","last_unstale":"2024-04-16T15:41:24.528233+0000","last_undegraded":"2024-04-16T15:41:24.528233+0000","last_fullsized":"2024-04-16T15:41:24.528233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528839+0000","last_change":"2024-04-16T15:41:23.521298+0000","last_active":"2024-04-16T15:41:24.528839+0000","last_peered":"2024-04-16T15:41:24.528839+0000","last_clean":"2024-04-16T15:41:24.528839+0000","last_became_active":"2024-04-16T15:41:23.521041+0000","last_became_peered":"2024-04-16T15:41:23.521041+0000","last_unstale":"2024-04-16T15:41:24.528839+0000","last_undegraded":"2024-04-16T15:41:24.528839+0000","last_fullsized":"2024-04-16T15:41:24.528839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:45.308062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683715+0000","last_change":"2024-04-16T15:41:22.529046+0000","last_active":"2024-04-16T15:41:24.683715+0000","last_peered":"2024-04-16T15:41:24.683715+0000","last_clean":"2024-04-16T15:41:24.683715+0000","last_became_active":"2024-04-16T15:41:22.528745+0000","last_became_peered":"2024-04-16T15:41:22.528745+0000","last_unstale":"2024-04-16T15:41:24.683715+0000","last_undegraded":"2024-04-16T15:41:24.683715+0000","last_fullsized":"2024-04-16T15:41:24.683715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:05.155726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683671+0000","last_change":"2024-04-16T15:41:23.522758+0000","last_active":"2024-04-16T15:41:24.683671+0000","last_peered":"2024-04-16T15:41:24.683671+0000","last_clean":"2024-04-16T15:41:24.683671+0000","last_became_active":"2024-04-16T15:41:23.521964+0000","last_became_peered":"2024-04-16T15:41:23.521964+0000","last_unstale":"2024-04-16T15:41:24.683671+0000","last_undegraded":"2024-04-16T15:41:24.683671+0000","last_fullsized":"2024-04-16T15:41:24.683671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070256+0000","last_change":"2024-04-16T15:41:22.523228+0000","last_active":"2024-04-16T15:41:25.070256+0000","last_peered":"2024-04-16T15:41:25.070256+0000","last_clean":"2024-04-16T15:41:25.070256+0000","last_became_active":"2024-04-16T15:41:22.522888+0000","last_became_peered":"2024-04-16T15:41:22.522888+0000","last_unstale":"2024-04-16T15:41:25.070256+0000","last_undegraded":"2024-04-16T15:41:25.070256+0000","last_fullsized":"2024-04-16T15:41:25.070256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:59.195492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069536+0000","last_change":"2024-04-16T15:41:23.525797+0000","last_active":"2024-04-16T15:41:25.069536+0000","last_peered":"2024-04-16T15:41:25.069536+0000","last_clean":"2024-04-16T15:41:25.069536+0000","last_became_active":"2024-04-16T15:41:23.525537+0000","last_became_peered":"2024-04-16T15:41:23.525537+0000","last_unstale":"2024-04-16T15:41:25.069536+0000","last_undegraded":"2024-04-16T15:41:25.069536+0000","last_fullsized":"2024-04-16T15:41:25.069536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:06:06.161198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068884+0000","last_change":"2024-04-16T15:41:22.521885+0000","last_active":"2024-04-16T15:41:25.068884+0000","last_peered":"2024-04-16T15:41:25.068884+0000","last_clean":"2024-04-16T15:41:25.068884+0000","last_became_active":"2024-04-16T15:41:22.521574+0000","last_became_peered":"2024-04-16T15:41:22.521574+0000","last_unstale":"2024-04-16T15:41:25.068884+0000","last_undegraded":"2024-04-16T15:41:25.068884+0000","last_fullsized":"2024-04-16T15:41:25.068884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:19:59.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527576+0000","last_change":"2024-04-16T15:41:23.529208+0000","last_active":"2024-04-16T15:41:24.527576+0000","last_peered":"2024-04-16T15:41:24.527576+0000","last_clean":"2024-04-16T15:41:24.527576+0000","last_became_active":"2024-04-16T15:41:23.528946+0000","last_became_peered":"2024-04-16T15:41:23.528946+0000","last_unstale":"2024-04-16T15:41:24.527576+0000","last_undegraded":"2024-04-16T15:41:24.527576+0000","last_fullsized":"2024-04-16T15:41:24.527576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:59:03.633363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069471+0000","last_change":"2024-04-16T15:41:22.525515+0000","last_active":"2024-04-16T15:41:25.069471+0000","last_peered":"2024-04-16T15:41:25.069471+0000","last_clean":"2024-04-16T15:41:25.069471+0000","last_became_active":"2024-04-16T15:41:22.525228+0000","last_became_peered":"2024-04-16T15:41:22.525228+0000","last_unstale":"2024-04-16T15:41:25.069471+0000","last_undegraded":"2024-04-16T15:41:25.069471+0000","last_fullsized":"2024-04-16T15:41:25.069471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:55:38.421964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068802+0000","last_change":"2024-04-16T15:41:23.522196+0000","last_active":"2024-04-16T15:41:25.068802+0000","last_peered":"2024-04-16T15:41:25.068802+0000","last_clean":"2024-04-16T15:41:25.068802+0000","last_became_active":"2024-04-16T15:41:23.521931+0000","last_became_peered":"2024-04-16T15:41:23.521931+0000","last_unstale":"2024-04-16T15:41:25.068802+0000","last_undegraded":"2024-04-16T15:41:25.068802+0000","last_fullsized":"2024-04-16T15:41:25.068802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069160+0000","last_change":"2024-04-16T15:41:22.523743+0000","last_active":"2024-04-16T15:41:25.069160+0000","last_peered":"2024-04-16T15:41:25.069160+0000","last_clean":"2024-04-16T15:41:25.069160+0000","last_became_active":"2024-04-16T15:41:22.523348+0000","last_became_peered":"2024-04-16T15:41:22.523348+0000","last_unstale":"2024-04-16T15:41:25.069160+0000","last_undegraded":"2024-04-16T15:41:25.069160+0000","last_fullsized":"2024-04-16T15:41:25.069160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:48.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528766+0000","last_change":"2024-04-16T15:41:23.524838+0000","last_active":"2024-04-16T15:41:24.528766+0000","last_peered":"2024-04-16T15:41:24.528766+0000","last_clean":"2024-04-16T15:41:24.528766+0000","last_became_active":"2024-04-16T15:41:23.524540+0000","last_became_peered":"2024-04-16T15:41:23.524540+0000","last_unstale":"2024-04-16T15:41:24.528766+0000","last_undegraded":"2024-04-16T15:41:24.528766+0000","last_fullsized":"2024-04-16T15:41:24.528766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:02.367931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527606+0000","last_change":"2024-04-16T15:41:22.522239+0000","last_active":"2024-04-16T15:41:24.527606+0000","last_peered":"2024-04-16T15:41:24.527606+0000","last_clean":"2024-04-16T15:41:24.527606+0000","last_became_active":"2024-04-16T15:41:22.521860+0000","last_became_peered":"2024-04-16T15:41:22.521860+0000","last_unstale":"2024-04-16T15:41:24.527606+0000","last_undegraded":"2024-04-16T15:41:24.527606+0000","last_fullsized":"2024-04-16T15:41:24.527606+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:18.832306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530187+0000","last_change":"2024-04-16T15:41:23.522134+0000","last_active":"2024-04-16T15:41:24.530187+0000","last_peered":"2024-04-16T15:41:24.530187+0000","last_clean":"2024-04-16T15:41:24.530187+0000","last_became_active":"2024-04-16T15:41:23.521852+0000","last_became_peered":"2024-04-16T15:41:23.521852+0000","last_unstale":"2024-04-16T15:41:24.530187+0000","last_undegraded":"2024-04-16T15:41:24.530187+0000","last_fullsized":"2024-04-16T15:41:24.530187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:46.030184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069255+0000","last_change":"2024-04-16T15:41:22.525713+0000","last_active":"2024-04-16T15:41:25.069255+0000","last_peered":"2024-04-16T15:41:25.069255+0000","last_clean":"2024-04-16T15:41:25.069255+0000","last_became_active":"2024-04-16T15:41:22.525423+0000","last_became_peered":"2024-04-16T15:41:22.525423+0000","last_unstale":"2024-04-16T15:41:25.069255+0000","last_undegraded":"2024-04-16T15:41:25.069255+0000","last_fullsized":"2024-04-16T15:41:25.069255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:03:22.255097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068828+0000","last_change":"2024-04-16T15:41:23.536118+0000","last_active":"2024-04-16T15:41:25.068828+0000","last_peered":"2024-04-16T15:41:25.068828+0000","last_clean":"2024-04-16T15:41:25.068828+0000","last_became_active":"2024-04-16T15:41:23.535863+0000","last_became_peered":"2024-04-16T15:41:23.535863+0000","last_unstale":"2024-04-16T15:41:25.068828+0000","last_undegraded":"2024-04-16T15:41:25.068828+0000","last_fullsized":"2024-04-16T15:41:25.068828+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:16.846698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528405+0000","last_change":"2024-04-16T15:41:22.525073+0000","last_active":"2024-04-16T15:41:24.528405+0000","last_peered":"2024-04-16T15:41:24.528405+0000","last_clean":"2024-04-16T15:41:24.528405+0000","last_became_active":"2024-04-16T15:41:22.524464+0000","last_became_peered":"2024-04-16T15:41:22.524464+0000","last_unstale":"2024-04-16T15:41:24.528405+0000","last_undegraded":"2024-04-16T15:41:24.528405+0000","last_fullsized":"2024-04-16T15:41:24.528405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:17.551990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529022+0000","last_change":"2024-04-16T15:41:23.526852+0000","last_active":"2024-04-16T15:41:24.529022+0000","last_peered":"2024-04-16T15:41:24.529022+0000","last_clean":"2024-04-16T15:41:24.529022+0000","last_became_active":"2024-04-16T15:41:23.526580+0000","last_became_peered":"2024-04-16T15:41:23.526580+0000","last_unstale":"2024-04-16T15:41:24.529022+0000","last_undegraded":"2024-04-16T15:41:24.529022+0000","last_fullsized":"2024-04-16T15:41:24.529022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070264+0000","last_change":"2024-04-16T15:41:22.523803+0000","last_active":"2024-04-16T15:41:25.070264+0000","last_peered":"2024-04-16T15:41:25.070264+0000","last_clean":"2024-04-16T15:41:25.070264+0000","last_became_active":"2024-04-16T15:41:22.523524+0000","last_became_peered":"2024-04-16T15:41:22.523524+0000","last_unstale":"2024-04-16T15:41:25.070264+0000","last_undegraded":"2024-04-16T15:41:25.070264+0000","last_fullsized":"2024-04-16T15:41:25.070264+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:18:46.457169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.530098+0000","last_change":"2024-04-16T15:41:23.521456+0000","last_active":"2024-04-16T15:41:24.530098+0000","last_peered":"2024-04-16T15:41:24.530098+0000","last_clean":"2024-04-16T15:41:24.530098+0000","last_became_active":"2024-04-16T15:41:23.521114+0000","last_became_peered":"2024-04-16T15:41:23.521114+0000","last_unstale":"2024-04-16T15:41:24.530098+0000","last_undegraded":"2024-04-16T15:41:24.530098+0000","last_fullsized":"2024-04-16T15:41:24.530098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:12:45.645892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069158+0000","last_change":"2024-04-16T15:41:22.517077+0000","last_active":"2024-04-16T15:41:25.069158+0000","last_peered":"2024-04-16T15:41:25.069158+0000","last_clean":"2024-04-16T15:41:25.069158+0000","last_became_active":"2024-04-16T15:41:22.516776+0000","last_became_peered":"2024-04-16T15:41:22.516776+0000","last_unstale":"2024-04-16T15:41:25.069158+0000","last_undegraded":"2024-04-16T15:41:25.069158+0000","last_fullsized":"2024-04-16T15:41:25.069158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070000+0000","last_change":"2024-04-16T15:41:23.521606+0000","last_active":"2024-04-16T15:41:25.070000+0000","last_peered":"2024-04-16T15:41:25.070000+0000","last_clean":"2024-04-16T15:41:25.070000+0000","last_became_active":"2024-04-16T15:41:23.521351+0000","last_became_peered":"2024-04-16T15:41:23.521351+0000","last_unstale":"2024-04-16T15:41:25.070000+0000","last_undegraded":"2024-04-16T15:41:25.070000+0000","last_fullsized":"2024-04-16T15:41:25.070000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.069018+0000","last_change":"2024-04-16T15:41:22.517838+0000","last_active":"2024-04-16T15:41:25.069018+0000","last_peered":"2024-04-16T15:41:25.069018+0000","last_clean":"2024-04-16T15:41:25.069018+0000","last_became_active":"2024-04-16T15:41:22.517494+0000","last_became_peered":"2024-04-16T15:41:22.517494+0000","last_unstale":"2024-04-16T15:41:25.069018+0000","last_undegraded":"2024-04-16T15:41:25.069018+0000","last_fullsized":"2024-04-16T15:41:25.069018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:39:50.605821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.070060+0000","last_change":"2024-04-16T15:41:23.524122+0000","last_active":"2024-04-16T15:41:25.070060+0000","last_peered":"2024-04-16T15:41:25.070060+0000","last_clean":"2024-04-16T15:41:25.070060+0000","last_became_active":"2024-04-16T15:41:23.523821+0000","last_became_peered":"2024-04-16T15:41:23.523821+0000","last_unstale":"2024-04-16T15:41:25.070060+0000","last_undegraded":"2024-04-16T15:41:25.070060+0000","last_fullsized":"2024-04-16T15:41:25.070060+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:25.931345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:25.068589+0000","last_change":"2024-04-16T15:41:22.519275+0000","last_active":"2024-04-16T15:41:25.068589+0000","last_peered":"2024-04-16T15:41:25.068589+0000","last_clean":"2024-04-16T15:41:25.068589+0000","last_became_active":"2024-04-16T15:41:22.518999+0000","last_became_peered":"2024-04-16T15:41:22.518999+0000","last_unstale":"2024-04-16T15:41:25.068589+0000","last_undegraded":"2024-04-16T15:41:25.068589+0000","last_fullsized":"2024-04-16T15:41:25.068589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:08:32.722620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683431+0000","last_change":"2024-04-16T15:41:23.522582+0000","last_active":"2024-04-16T15:41:24.683431+0000","last_peered":"2024-04-16T15:41:24.683431+0000","last_clean":"2024-04-16T15:41:24.683431+0000","last_became_active":"2024-04-16T15:41:23.522254+0000","last_became_peered":"2024-04-16T15:41:23.522254+0000","last_unstale":"2024-04-16T15:41:24.683431+0000","last_undegraded":"2024-04-16T15:41:24.683431+0000","last_fullsized":"2024-04-16T15:41:24.683431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:28:04.795881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.683270+0000","last_change":"2024-04-16T15:41:22.525440+0000","last_active":"2024-04-16T15:41:24.683270+0000","last_peered":"2024-04-16T15:41:24.683270+0000","last_clean":"2024-04-16T15:41:24.683270+0000","last_became_active":"2024-04-16T15:41:22.523261+0000","last_became_peered":"2024-04-16T15:41:22.523261+0000","last_unstale":"2024-04-16T15:41:24.683270+0000","last_undegraded":"2024-04-16T15:41:24.683270+0000","last_fullsized":"2024-04-16T15:41:24.683270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:35:47.601870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528023+0000","last_change":"2024-04-16T15:41:23.526801+0000","last_active":"2024-04-16T15:41:24.528023+0000","last_peered":"2024-04-16T15:41:24.528023+0000","last_clean":"2024-04-16T15:41:24.528023+0000","last_became_active":"2024-04-16T15:41:23.526499+0000","last_became_peered":"2024-04-16T15:41:23.526499+0000","last_unstale":"2024-04-16T15:41:24.528023+0000","last_undegraded":"2024-04-16T15:41:24.528023+0000","last_fullsized":"2024-04-16T15:41:24.528023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:38.150207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.529584+0000","last_change":"2024-04-16T15:41:22.521239+0000","last_active":"2024-04-16T15:41:24.529584+0000","last_peered":"2024-04-16T15:41:24.529584+0000","last_clean":"2024-04-16T15:41:24.529584+0000","last_became_active":"2024-04-16T15:41:22.520944+0000","last_became_peered":"2024-04-16T15:41:22.520944+0000","last_unstale":"2024-04-16T15:41:24.529584+0000","last_undegraded":"2024-04-16T15:41:24.529584+0000","last_fullsized":"2024-04-16T15:41:24.529584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:37.008786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528112+0000","last_change":"2024-04-16T15:41:23.532440+0000","last_active":"2024-04-16T15:41:24.528112+0000","last_peered":"2024-04-16T15:41:24.528112+0000","last_clean":"2024-04-16T15:41:24.528112+0000","last_became_active":"2024-04-16T15:41:23.530755+0000","last_became_peered":"2024-04-16T15:41:23.530755+0000","last_unstale":"2024-04-16T15:41:24.528112+0000","last_undegraded":"2024-04-16T15:41:24.528112+0000","last_fullsized":"2024-04-16T15:41:24.528112+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:22.498925+0000","last_clean_scrub_stamp":"2024-04-16T15:41:22.498925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:31.174426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.528231+0000","last_change":"2024-04-16T15:41:22.523523+0000","last_active":"2024-04-16T15:41:24.528231+0000","last_peered":"2024-04-16T15:41:24.528231+0000","last_clean":"2024-04-16T15:41:24.528231+0000","last_became_active":"2024-04-16T15:41:22.522766+0000","last_became_peered":"2024-04-16T15:41:22.522766+0000","last_unstale":"2024-04-16T15:41:24.528231+0000","last_undegraded":"2024-04-16T15:41:24.528231+0000","last_fullsized":"2024-04-16T15:41:24.528231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:21.493453+0000","last_clean_scrub_stamp":"2024-04-16T15:41:21.493453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:27.943133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-16T15:41:24.527623+0000","last_change":"2024-04-16T15:40:12.959421+0000","last_active":"2024-04-16T15:41:24.527623+0000","last_peered":"2024-04-16T15:41:24.527623+0000","last_clean":"2024-04-16T15:41:24.527623+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:41:24.527623+0000","last_undegraded":"2024-04-16T15:41:24.527623+0000","last_fullsized":"2024-04-16T15:41:24.527623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:40:11.754985+0000","last_clean_scrub_stamp":"2024-04-16T15:40:11.754985+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:53:19.374061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":782248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":191844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":780600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:41:35.722 INFO:tasks.ceph:pgid 6.3f last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.722 INFO:tasks.ceph:pgid 6.3e last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.722 INFO:tasks.ceph:pgid 6.3d last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.722 INFO:tasks.ceph:pgid 6.3c last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.723 INFO:tasks.ceph:pgid 6.3b last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.723 INFO:tasks.ceph:pgid 6.3a last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.723 INFO:tasks.ceph:pgid 6.39 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.723 INFO:tasks.ceph:pgid 6.38 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.723 INFO:tasks.ceph:pgid 6.37 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.723 INFO:tasks.ceph:pgid 6.36 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.724 INFO:tasks.ceph:pgid 6.35 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.724 INFO:tasks.ceph:pgid 6.34 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.724 INFO:tasks.ceph:pgid 6.33 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.724 INFO:tasks.ceph:pgid 6.32 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.724 INFO:tasks.ceph:pgid 6.31 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.724 INFO:tasks.ceph:pgid 6.30 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.724 INFO:tasks.ceph:pgid 6.2f last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.725 INFO:tasks.ceph:pgid 6.2e last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.725 INFO:tasks.ceph:pgid 6.2d last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.725 INFO:tasks.ceph:pgid 6.2c last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.725 INFO:tasks.ceph:pgid 6.2b last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.725 INFO:tasks.ceph:pgid 6.2a last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.725 INFO:tasks.ceph:pgid 6.29 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.725 INFO:tasks.ceph:pgid 6.28 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.726 INFO:tasks.ceph:pgid 6.27 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.726 INFO:tasks.ceph:pgid 6.26 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.726 INFO:tasks.ceph:pgid 6.25 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.726 INFO:tasks.ceph:pgid 6.24 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.726 INFO:tasks.ceph:pgid 6.23 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.726 INFO:tasks.ceph:pgid 6.22 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.727 INFO:tasks.ceph:pgid 6.21 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.727 INFO:tasks.ceph:pgid 6.20 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.727 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.727 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.727 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.727 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.727 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.728 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.728 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.728 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.728 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.728 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.728 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.729 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.729 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.729 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.729 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.729 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.729 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.730 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.730 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.730 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.730 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.730 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.730 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.730 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.731 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.731 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.731 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.731 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.731 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.731 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.732 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.732 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.732 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.732 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.732 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.732 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.732 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.733 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.733 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.733 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.733 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.733 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.733 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.734 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.734 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.734 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.734 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.734 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.734 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.734 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.735 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.735 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.735 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.735 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.735 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.736 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.736 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.736 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.736 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.736 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.736 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.736 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.737 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-16T15:41:22.498925+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=22, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.737 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-16T15:41:21.493453+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=21, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.737 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-16T15:40:11.754985+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=40, tm_sec=11, tm_wday=1, tm_yday=107, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=16, tm_hour=15, tm_min=41, tm_sec=30, tm_wday=1, tm_yday=107, tm_isdst=0) 2024-04-16T15:41:35.737 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-16T15:41:37.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-04-16T15:41:39.274 INFO:tasks.ceph.mds.b.smithi122.stderr:2024-04-16T15:41:39.275+0000 7f25e9893640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-04-16T15:41:43.484 INFO:tasks.ceph.mds.b.smithi122.stderr:2024-04-16T15:41:43.485+0000 7f25ed89b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-16T15:41:43.970 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-04-16T15:41:50.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2024-04-16T15:41:53.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:41:53.816 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:41:55.739 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T15:41:55.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.855+0000 7f892ac14640 1 Processor -- start 2024-04-16T15:41:55.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.855+0000 7f892ac14640 1 -- start start 2024-04-16T15:41:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f892ac14640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f89240ab950 0x7f89240aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f892ac14640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8924153ba0 con 0x7f8924150f40 2024-04-16T15:41:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f892ac14640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8924153900 con 0x7f89240aa900 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f892ac14640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7f89240ad3e0 con 0x7f89240ab950 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f8923fff640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7f89240ab950 0x7f89240aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f89289b0640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f8924150f40 0x7f89240accd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:44188/0 (socket says 172.21.15.71:44188) 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f89289b0640 1 -- 172.21.15.71:0/3928527075 learned_addr learned my addr 172.21.15.71:0/3928527075 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f8923fff640 1 -- 172.21.15.71:0/3928527075 >> v1:172.21.15.122:6789/0 conn(0x7f89240aa900 legacy=0x7f892415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f8923fff640 1 -- 172.21.15.71:0/3928527075 >> v1:172.21.15.71:6789/0 conn(0x7f8924150f40 legacy=0x7f89240accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f8923fff640 1 -- 172.21.15.71:0/3928527075 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89240ac4d0 con 0x7f89240ab950 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.856+0000 7f89237fe640 1 -- 172.21.15.71:0/3928527075 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430370403 0 0) 0x7f8924153ba0 con 0x7f8924150f40 2024-04-16T15:41:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.857+0000 7f8923fff640 1 --2- 172.21.15.71:0/3928527075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89240aa3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f89100097a0 tx=0x7f891002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae71e66cb5e3e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.857+0000 7f89237fe640 1 -- 172.21.15.71:0/3928527075 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910034070 con 0x7f89240ab950 2024-04-16T15:41:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.857+0000 7f89237fe640 1 -- 172.21.15.71:0/3928527075 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8910030610 con 0x7f89240ab950 2024-04-16T15:41:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.857+0000 7f89237fe640 1 -- 172.21.15.71:0/3928527075 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89100309e0 con 0x7f89240ab950 2024-04-16T15:41:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.857+0000 7f892ac14640 1 -- 172.21.15.71:0/3928527075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 msgr2=0x7f89240aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.857+0000 7f892ac14640 1 --2- 172.21.15.71:0/3928527075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89240aa3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f89100097a0 tx=0x7f891002b960 comp rx=0 tx=0).stop 2024-04-16T15:41:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.858+0000 7f892ac14640 1 -- 172.21.15.71:0/3928527075 shutdown_connections 2024-04-16T15:41:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.858+0000 7f892ac14640 1 --2- 172.21.15.71:0/3928527075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89240aa3c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.858+0000 7f892ac14640 1 -- 172.21.15.71:0/3928527075 >> 172.21.15.71:0/3928527075 conn(0x7f89240c3e10 msgr2=0x7f89240c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:55.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.858+0000 7f892ac14640 1 -- 172.21.15.71:0/3928527075 shutdown_connections 2024-04-16T15:41:55.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.858+0000 7f892ac14640 1 -- 172.21.15.71:0/3928527075 wait complete. 2024-04-16T15:41:55.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.859+0000 7f892ac14640 1 Processor -- start 2024-04-16T15:41:55.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.859+0000 7f892ac14640 1 -- start start 2024-04-16T15:41:55.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f892ac14640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89240aa900 0x7f8924146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:55.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f892ac14640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89241e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:55.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f892ac14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924150f40 0x7f89241ea020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:55.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f892ac14640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f892406fdd0 con 0x7f89240ab950 2024-04-16T15:41:55.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f892ac14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89240c1f20 con 0x7f8924150f40 2024-04-16T15:41:55.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f892ac14640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f892406ff50 con 0x7f89240aa900 2024-04-16T15:41:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89241e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89241e9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42704/0 (socket says 172.21.15.71:42704) 2024-04-16T15:41:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 -- 172.21.15.71:0/408070430 learned_addr learned my addr 172.21.15.71:0/408070430 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f89289b0640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89240aa900 0x7f8924146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f89291b1640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924150f40 0x7f89241ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 -- 172.21.15.71:0/408070430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89240aa900 msgr2=0x7f8924146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89240aa900 0x7f8924146d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 -- 172.21.15.71:0/408070430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924150f40 msgr2=0x7f89241ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924150f40 0x7f89241ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 -- 172.21.15.71:0/408070430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241e9620 con 0x7f89240ab950 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f8923fff640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89241e9ae0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f89100098c0 tx=0x7f89100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f89217fa640 1 -- 172.21.15.71:0/408070430 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910034040 con 0x7f89240ab950 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.860+0000 7f89217fa640 1 -- 172.21.15.71:0/408070430 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8910004030 con 0x7f89240ab950 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.861+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241489b0 con 0x7f89240ab950 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.861+0000 7f89217fa640 1 -- 172.21.15.71:0/408070430 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910004420 con 0x7f89240ab950 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.861+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241ed310 con 0x7f89240ab950 2024-04-16T15:41:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.861+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8924068d20 con 0x7f89240ab950 2024-04-16T15:41:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.862+0000 7f89217fa640 1 -- 172.21.15.71:0/408070430 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8910037070 con 0x7f89240ab950 2024-04-16T15:41:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.862+0000 7f89217fa640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f890806c8f0 0x7f890806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.863+0000 7f89217fa640 1 -- 172.21.15.71:0/408070430 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f89100ad940 con 0x7f89240ab950 2024-04-16T15:41:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.863+0000 7f89289b0640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f890806c8f0 0x7f890806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.863+0000 7f89289b0640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f890806c8f0 0x7f890806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8918007fd0 tx=0x7f8918007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:55.867+0000 7f89217fa640 1 -- 172.21.15.71:0/408070430 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89241e9620 con 0x7f89240ab950 2024-04-16T15:41:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.040+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 --> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89241ea560 con 0x7f890806c8f0 2024-04-16T15:41:56.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.046+0000 7f89217fa640 1 -- 172.21.15.71:0/408070430 <== mgr.4107 v2:172.21.15.71:6832/4207135910 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251894 (secure 0 0 0) 0x7f89241ea560 con 0x7f890806c8f0 2024-04-16T15:41:56.047 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:56.048 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-16T15:41:56.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.048+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f890806c8f0 msgr2=0x7f890806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.048+0000 7f892ac14640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f890806c8f0 0x7f890806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8918007fd0 tx=0x7f8918007de0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.048+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 msgr2=0x7f89241e9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.048+0000 7f892ac14640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89241e9ae0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f89100098c0 tx=0x7f89100046c0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.049+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 shutdown_connections 2024-04-16T15:41:56.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.049+0000 7f892ac14640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f890806c8f0 0x7f890806edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.049+0000 7f892ac14640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924150f40 0x7f89241ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.049+0000 7f892ac14640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89240ab950 0x7f89241e9ae0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.049+0000 7f892ac14640 1 --2- 172.21.15.71:0/408070430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89240aa900 0x7f8924146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.049+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 >> 172.21.15.71:0/408070430 conn(0x7f89240c3e10 msgr2=0x7f8924153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:56.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.049+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 shutdown_connections 2024-04-16T15:41:56.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.049+0000 7f892ac14640 1 -- 172.21.15.71:0/408070430 wait complete. 2024-04-16T15:41:56.058 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-16T15:41:55.719371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":193,"num_read_kb":180,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220876,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753844,"statfs":{"total":773094113280,"available":772867936256,"internally_reserved":0,"allocated":4857856,"data_stored":2758924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":70,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002889"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.771280+0000","last_change":"2024-04-16T15:41:42.771318+0000","last_active":"2024-04-16T15:41:42.771280+0000","last_peered":"2024-04-16T15:41:42.771280+0000","last_clean":"2024-04-16T15:41:42.771280+0000","last_became_active":"2024-04-16T15:41:23.522804+0000","last_became_peered":"2024-04-16T15:41:23.522804+0000","last_unstale":"2024-04-16T15:41:42.771280+0000","last_undegraded":"2024-04-16T15:41:42.771280+0000","last_fullsized":"2024-04-16T15:41:42.771280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:42.771198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:42.771198+0000","last_clean_scrub_stamp":"2024-04-16T15:41:42.771198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:29:23.092765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.270928+0000","last_change":"2024-04-16T15:41:37.858008+0000","last_active":"2024-04-16T15:41:39.270928+0000","last_peered":"2024-04-16T15:41:39.270928+0000","last_clean":"2024-04-16T15:41:39.270928+0000","last_became_active":"2024-04-16T15:41:23.517372+0000","last_became_peered":"2024-04-16T15:41:23.517372+0000","last_unstale":"2024-04-16T15:41:39.270928+0000","last_undegraded":"2024-04-16T15:41:39.270928+0000","last_fullsized":"2024-04-16T15:41:39.270928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:37.857899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:37.857899+0000","last_clean_scrub_stamp":"2024-04-16T15:41:37.857899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:00:34.190859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:40.961746+0000","last_change":"2024-04-16T15:41:40.961779+0000","last_active":"2024-04-16T15:41:40.961746+0000","last_peered":"2024-04-16T15:41:40.961746+0000","last_clean":"2024-04-16T15:41:40.961746+0000","last_became_active":"2024-04-16T15:41:23.533520+0000","last_became_peered":"2024-04-16T15:41:23.533520+0000","last_unstale":"2024-04-16T15:41:40.961746+0000","last_undegraded":"2024-04-16T15:41:40.961746+0000","last_fullsized":"2024-04-16T15:41:40.961746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:40.961670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:40.961670+0000","last_clean_scrub_stamp":"2024-04-16T15:41:40.961670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:19.933650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:41.147600+0000","last_change":"2024-04-16T15:41:41.147624+0000","last_active":"2024-04-16T15:41:41.147600+0000","last_peered":"2024-04-16T15:41:41.147600+0000","last_clean":"2024-04-16T15:41:41.147600+0000","last_became_active":"2024-04-16T15:41:23.521419+0000","last_became_peered":"2024-04-16T15:41:23.521419+0000","last_unstale":"2024-04-16T15:41:41.147600+0000","last_undegraded":"2024-04-16T15:41:41.147600+0000","last_fullsized":"2024-04-16T15:41:41.147600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:41.147516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:41.147516+0000","last_clean_scrub_stamp":"2024-04-16T15:41:41.147516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:17:15.751537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.124400+0000","last_change":"2024-04-16T15:41:42.124439+0000","last_active":"2024-04-16T15:41:42.124400+0000","last_peered":"2024-04-16T15:41:42.124400+0000","last_clean":"2024-04-16T15:41:42.124400+0000","last_became_active":"2024-04-16T15:41:23.522009+0000","last_became_peered":"2024-04-16T15:41:23.522009+0000","last_unstale":"2024-04-16T15:41:42.124400+0000","last_undegraded":"2024-04-16T15:41:42.124400+0000","last_fullsized":"2024-04-16T15:41:42.124400+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:42.124298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:42.124298+0000","last_clean_scrub_stamp":"2024-04-16T15:41:42.124298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:50.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.279582+0000","last_change":"2024-04-16T15:41:37.701477+0000","last_active":"2024-04-16T15:41:39.279582+0000","last_peered":"2024-04-16T15:41:39.279582+0000","last_clean":"2024-04-16T15:41:39.279582+0000","last_became_active":"2024-04-16T15:41:23.522742+0000","last_became_peered":"2024-04-16T15:41:23.522742+0000","last_unstale":"2024-04-16T15:41:39.279582+0000","last_undegraded":"2024-04-16T15:41:39.279582+0000","last_fullsized":"2024-04-16T15:41:39.279582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:37.701352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:37.701352+0000","last_clean_scrub_stamp":"2024-04-16T15:41:37.701352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:17:02.303313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:46.045590+0000","last_change":"2024-04-16T15:41:46.045614+0000","last_active":"2024-04-16T15:41:46.045590+0000","last_peered":"2024-04-16T15:41:46.045590+0000","last_clean":"2024-04-16T15:41:46.045590+0000","last_became_active":"2024-04-16T15:41:23.523022+0000","last_became_peered":"2024-04-16T15:41:23.523022+0000","last_unstale":"2024-04-16T15:41:46.045590+0000","last_undegraded":"2024-04-16T15:41:46.045590+0000","last_fullsized":"2024-04-16T15:41:46.045590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:46.045506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:46.045506+0000","last_clean_scrub_stamp":"2024-04-16T15:41:46.045506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:46:13.205565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:46.722285+0000","last_change":"2024-04-16T15:41:46.722313+0000","last_active":"2024-04-16T15:41:46.722285+0000","last_peered":"2024-04-16T15:41:46.722285+0000","last_clean":"2024-04-16T15:41:46.722285+0000","last_became_active":"2024-04-16T15:41:23.535156+0000","last_became_peered":"2024-04-16T15:41:23.535156+0000","last_unstale":"2024-04-16T15:41:46.722285+0000","last_undegraded":"2024-04-16T15:41:46.722285+0000","last_fullsized":"2024-04-16T15:41:46.722285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:46.722194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:46.722194+0000","last_clean_scrub_stamp":"2024-04-16T15:41:46.722194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:21:15.999438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:41.915395+0000","last_change":"2024-04-16T15:41:41.915423+0000","last_active":"2024-04-16T15:41:41.915395+0000","last_peered":"2024-04-16T15:41:41.915395+0000","last_clean":"2024-04-16T15:41:41.915395+0000","last_became_active":"2024-04-16T15:41:23.534726+0000","last_became_peered":"2024-04-16T15:41:23.534726+0000","last_unstale":"2024-04-16T15:41:41.915395+0000","last_undegraded":"2024-04-16T15:41:41.915395+0000","last_fullsized":"2024-04-16T15:41:41.915395+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:41.915292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:41.915292+0000","last_clean_scrub_stamp":"2024-04-16T15:41:41.915292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:38:48.311925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:45.043968+0000","last_change":"2024-04-16T15:41:45.043997+0000","last_active":"2024-04-16T15:41:45.043968+0000","last_peered":"2024-04-16T15:41:45.043968+0000","last_clean":"2024-04-16T15:41:45.043968+0000","last_became_active":"2024-04-16T15:41:23.522437+0000","last_became_peered":"2024-04-16T15:41:23.522437+0000","last_unstale":"2024-04-16T15:41:45.043968+0000","last_undegraded":"2024-04-16T15:41:45.043968+0000","last_fullsized":"2024-04-16T15:41:45.043968+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:45.043868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:45.043868+0000","last_clean_scrub_stamp":"2024-04-16T15:41:45.043868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:29:25.365435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:49.103858+0000","last_change":"2024-04-16T15:41:49.103905+0000","last_active":"2024-04-16T15:41:49.103858+0000","last_peered":"2024-04-16T15:41:49.103858+0000","last_clean":"2024-04-16T15:41:49.103858+0000","last_became_active":"2024-04-16T15:41:23.530320+0000","last_became_peered":"2024-04-16T15:41:23.530320+0000","last_unstale":"2024-04-16T15:41:49.103858+0000","last_undegraded":"2024-04-16T15:41:49.103858+0000","last_fullsized":"2024-04-16T15:41:49.103858+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:49.103745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:49.103745+0000","last_clean_scrub_stamp":"2024-04-16T15:41:49.103745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:02:37.876372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.281975+0000","last_change":"2024-04-16T15:41:38.951009+0000","last_active":"2024-04-16T15:41:39.281975+0000","last_peered":"2024-04-16T15:41:39.281975+0000","last_clean":"2024-04-16T15:41:39.281975+0000","last_became_active":"2024-04-16T15:41:23.523081+0000","last_became_peered":"2024-04-16T15:41:23.523081+0000","last_unstale":"2024-04-16T15:41:39.281975+0000","last_undegraded":"2024-04-16T15:41:39.281975+0000","last_fullsized":"2024-04-16T15:41:39.281975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:38.950898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:38.950898+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.950898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:44:28.549087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:44.052453+0000","last_change":"2024-04-16T15:41:44.052492+0000","last_active":"2024-04-16T15:41:44.052453+0000","last_peered":"2024-04-16T15:41:44.052453+0000","last_clean":"2024-04-16T15:41:44.052453+0000","last_became_active":"2024-04-16T15:41:23.521531+0000","last_became_peered":"2024-04-16T15:41:23.521531+0000","last_unstale":"2024-04-16T15:41:44.052453+0000","last_undegraded":"2024-04-16T15:41:44.052453+0000","last_fullsized":"2024-04-16T15:41:44.052453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:44.052366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:44.052366+0000","last_clean_scrub_stamp":"2024-04-16T15:41:44.052366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:43.948064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:41.324977+0000","last_change":"2024-04-16T15:41:41.325005+0000","last_active":"2024-04-16T15:41:41.324977+0000","last_peered":"2024-04-16T15:41:41.324977+0000","last_clean":"2024-04-16T15:41:41.324977+0000","last_became_active":"2024-04-16T15:41:23.528557+0000","last_became_peered":"2024-04-16T15:41:23.528557+0000","last_unstale":"2024-04-16T15:41:41.324977+0000","last_undegraded":"2024-04-16T15:41:41.324977+0000","last_fullsized":"2024-04-16T15:41:41.324977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:41.324853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:41.324853+0000","last_clean_scrub_stamp":"2024-04-16T15:41:41.324853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:24.238297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.985676+0000","last_change":"2024-04-16T15:41:39.985697+0000","last_active":"2024-04-16T15:41:39.985676+0000","last_peered":"2024-04-16T15:41:39.985676+0000","last_clean":"2024-04-16T15:41:39.985676+0000","last_became_active":"2024-04-16T15:41:23.513340+0000","last_became_peered":"2024-04-16T15:41:23.513340+0000","last_unstale":"2024-04-16T15:41:39.985676+0000","last_undegraded":"2024-04-16T15:41:39.985676+0000","last_fullsized":"2024-04-16T15:41:39.985676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:39.985594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:39.985594+0000","last_clean_scrub_stamp":"2024-04-16T15:41:39.985594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:00:36.318554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:48.046827+0000","last_change":"2024-04-16T15:41:48.046850+0000","last_active":"2024-04-16T15:41:48.046827+0000","last_peered":"2024-04-16T15:41:48.046827+0000","last_clean":"2024-04-16T15:41:48.046827+0000","last_became_active":"2024-04-16T15:41:23.535549+0000","last_became_peered":"2024-04-16T15:41:23.535549+0000","last_unstale":"2024-04-16T15:41:48.046827+0000","last_undegraded":"2024-04-16T15:41:48.046827+0000","last_fullsized":"2024-04-16T15:41:48.046827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:48.046741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:48.046741+0000","last_clean_scrub_stamp":"2024-04-16T15:41:48.046741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:54:52.397664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:50.081113+0000","last_change":"2024-04-16T15:41:50.081139+0000","last_active":"2024-04-16T15:41:50.081113+0000","last_peered":"2024-04-16T15:41:50.081113+0000","last_clean":"2024-04-16T15:41:50.081113+0000","last_became_active":"2024-04-16T15:41:23.536009+0000","last_became_peered":"2024-04-16T15:41:23.536009+0000","last_unstale":"2024-04-16T15:41:50.081113+0000","last_undegraded":"2024-04-16T15:41:50.081113+0000","last_fullsized":"2024-04-16T15:41:50.081113+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:50.081020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:50.081020+0000","last_clean_scrub_stamp":"2024-04-16T15:41:50.081020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:52:53.933845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:41.102832+0000","last_change":"2024-04-16T15:41:41.102853+0000","last_active":"2024-04-16T15:41:41.102832+0000","last_peered":"2024-04-16T15:41:41.102832+0000","last_clean":"2024-04-16T15:41:41.102832+0000","last_became_active":"2024-04-16T15:41:23.523442+0000","last_became_peered":"2024-04-16T15:41:23.523442+0000","last_unstale":"2024-04-16T15:41:41.102832+0000","last_undegraded":"2024-04-16T15:41:41.102832+0000","last_fullsized":"2024-04-16T15:41:41.102832+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:41.102754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:41.102754+0000","last_clean_scrub_stamp":"2024-04-16T15:41:41.102754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:11:33.362109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.276025+0000","last_change":"2024-04-16T15:41:39.110964+0000","last_active":"2024-04-16T15:41:39.276025+0000","last_peered":"2024-04-16T15:41:39.276025+0000","last_clean":"2024-04-16T15:41:39.276025+0000","last_became_active":"2024-04-16T15:41:23.519258+0000","last_became_peered":"2024-04-16T15:41:23.519258+0000","last_unstale":"2024-04-16T15:41:39.276025+0000","last_undegraded":"2024-04-16T15:41:39.276025+0000","last_fullsized":"2024-04-16T15:41:39.276025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:39.110844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:39.110844+0000","last_clean_scrub_stamp":"2024-04-16T15:41:39.110844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:42.543263+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.271397+0000","last_change":"2024-04-16T15:41:38.019655+0000","last_active":"2024-04-16T15:41:39.271397+0000","last_peered":"2024-04-16T15:41:39.271397+0000","last_clean":"2024-04-16T15:41:39.271397+0000","last_became_active":"2024-04-16T15:41:23.521934+0000","last_became_peered":"2024-04-16T15:41:23.521934+0000","last_unstale":"2024-04-16T15:41:39.271397+0000","last_undegraded":"2024-04-16T15:41:39.271397+0000","last_fullsized":"2024-04-16T15:41:39.271397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:38.019549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:38.019549+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.019549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:46.695049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:41.035214+0000","last_change":"2024-04-16T15:41:41.035237+0000","last_active":"2024-04-16T15:41:41.035214+0000","last_peered":"2024-04-16T15:41:41.035214+0000","last_clean":"2024-04-16T15:41:41.035214+0000","last_became_active":"2024-04-16T15:41:23.520677+0000","last_became_peered":"2024-04-16T15:41:23.520677+0000","last_unstale":"2024-04-16T15:41:41.035214+0000","last_undegraded":"2024-04-16T15:41:41.035214+0000","last_fullsized":"2024-04-16T15:41:41.035214+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:41.035120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:41.035120+0000","last_clean_scrub_stamp":"2024-04-16T15:41:41.035120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:18:18.502538+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:40.290617+0000","last_change":"2024-04-16T15:41:40.290641+0000","last_active":"2024-04-16T15:41:40.290617+0000","last_peered":"2024-04-16T15:41:40.290617+0000","last_clean":"2024-04-16T15:41:40.290617+0000","last_became_active":"2024-04-16T15:41:23.527690+0000","last_became_peered":"2024-04-16T15:41:23.527690+0000","last_unstale":"2024-04-16T15:41:40.290617+0000","last_undegraded":"2024-04-16T15:41:40.290617+0000","last_fullsized":"2024-04-16T15:41:40.290617+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:40.290514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:40.290514+0000","last_clean_scrub_stamp":"2024-04-16T15:41:40.290514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:44:29.888702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:43.015279+0000","last_change":"2024-04-16T15:41:43.015312+0000","last_active":"2024-04-16T15:41:43.015279+0000","last_peered":"2024-04-16T15:41:43.015279+0000","last_clean":"2024-04-16T15:41:43.015279+0000","last_became_active":"2024-04-16T15:41:23.527372+0000","last_became_peered":"2024-04-16T15:41:23.527372+0000","last_unstale":"2024-04-16T15:41:43.015279+0000","last_undegraded":"2024-04-16T15:41:43.015279+0000","last_fullsized":"2024-04-16T15:41:43.015279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:43.015190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:43.015190+0000","last_clean_scrub_stamp":"2024-04-16T15:41:43.015190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:27:43.515487+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:43.156479+0000","last_change":"2024-04-16T15:41:43.156500+0000","last_active":"2024-04-16T15:41:43.156479+0000","last_peered":"2024-04-16T15:41:43.156479+0000","last_clean":"2024-04-16T15:41:43.156479+0000","last_became_active":"2024-04-16T15:41:23.524939+0000","last_became_peered":"2024-04-16T15:41:23.524939+0000","last_unstale":"2024-04-16T15:41:43.156479+0000","last_undegraded":"2024-04-16T15:41:43.156479+0000","last_fullsized":"2024-04-16T15:41:43.156479+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:43.156407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:43.156407+0000","last_clean_scrub_stamp":"2024-04-16T15:41:43.156407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:54.736377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.071973+0000","last_change":"2024-04-16T15:41:42.072016+0000","last_active":"2024-04-16T15:41:42.071973+0000","last_peered":"2024-04-16T15:41:42.071973+0000","last_clean":"2024-04-16T15:41:42.071973+0000","last_became_active":"2024-04-16T15:41:23.525999+0000","last_became_peered":"2024-04-16T15:41:23.525999+0000","last_unstale":"2024-04-16T15:41:42.071973+0000","last_undegraded":"2024-04-16T15:41:42.071973+0000","last_fullsized":"2024-04-16T15:41:42.071973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:42.071869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:42.071869+0000","last_clean_scrub_stamp":"2024-04-16T15:41:42.071869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:43:44.078249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.688837+0000","last_change":"2024-04-16T15:41:39.688863+0000","last_active":"2024-04-16T15:41:39.688837+0000","last_peered":"2024-04-16T15:41:39.688837+0000","last_clean":"2024-04-16T15:41:39.688837+0000","last_became_active":"2024-04-16T15:41:23.535192+0000","last_became_peered":"2024-04-16T15:41:23.535192+0000","last_unstale":"2024-04-16T15:41:39.688837+0000","last_undegraded":"2024-04-16T15:41:39.688837+0000","last_fullsized":"2024-04-16T15:41:39.688837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:39.688728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:39.688728+0000","last_clean_scrub_stamp":"2024-04-16T15:41:39.688728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:22.602172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:43.051372+0000","last_change":"2024-04-16T15:41:43.051398+0000","last_active":"2024-04-16T15:41:43.051372+0000","last_peered":"2024-04-16T15:41:43.051372+0000","last_clean":"2024-04-16T15:41:43.051372+0000","last_became_active":"2024-04-16T15:41:23.526765+0000","last_became_peered":"2024-04-16T15:41:23.526765+0000","last_unstale":"2024-04-16T15:41:43.051372+0000","last_undegraded":"2024-04-16T15:41:43.051372+0000","last_fullsized":"2024-04-16T15:41:43.051372+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:43.051286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:43.051286+0000","last_clean_scrub_stamp":"2024-04-16T15:41:43.051286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:17:14.102947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:48.059171+0000","last_change":"2024-04-16T15:41:48.059196+0000","last_active":"2024-04-16T15:41:48.059171+0000","last_peered":"2024-04-16T15:41:48.059171+0000","last_clean":"2024-04-16T15:41:48.059171+0000","last_became_active":"2024-04-16T15:41:23.525283+0000","last_became_peered":"2024-04-16T15:41:23.525283+0000","last_unstale":"2024-04-16T15:41:48.059171+0000","last_undegraded":"2024-04-16T15:41:48.059171+0000","last_fullsized":"2024-04-16T15:41:48.059171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:48.059090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:48.059090+0000","last_clean_scrub_stamp":"2024-04-16T15:41:48.059090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:38:52.959298+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:40.058343+0000","last_change":"2024-04-16T15:41:40.058379+0000","last_active":"2024-04-16T15:41:40.058343+0000","last_peered":"2024-04-16T15:41:40.058343+0000","last_clean":"2024-04-16T15:41:40.058343+0000","last_became_active":"2024-04-16T15:41:23.522762+0000","last_became_peered":"2024-04-16T15:41:23.522762+0000","last_unstale":"2024-04-16T15:41:40.058343+0000","last_undegraded":"2024-04-16T15:41:40.058343+0000","last_fullsized":"2024-04-16T15:41:40.058343+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:40.058232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:40.058232+0000","last_clean_scrub_stamp":"2024-04-16T15:41:40.058232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:03.205199+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.272518+0000","last_change":"2024-04-16T15:41:39.011822+0000","last_active":"2024-04-16T15:41:39.272518+0000","last_peered":"2024-04-16T15:41:39.272518+0000","last_clean":"2024-04-16T15:41:39.272518+0000","last_became_active":"2024-04-16T15:41:23.522179+0000","last_became_peered":"2024-04-16T15:41:23.522179+0000","last_unstale":"2024-04-16T15:41:39.272518+0000","last_undegraded":"2024-04-16T15:41:39.272518+0000","last_fullsized":"2024-04-16T15:41:39.272518+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:39.011724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:39.011724+0000","last_clean_scrub_stamp":"2024-04-16T15:41:39.011724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:05:50.591694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:45.031572+0000","last_change":"2024-04-16T15:41:45.031610+0000","last_active":"2024-04-16T15:41:45.031572+0000","last_peered":"2024-04-16T15:41:45.031572+0000","last_clean":"2024-04-16T15:41:45.031572+0000","last_became_active":"2024-04-16T15:41:23.528304+0000","last_became_peered":"2024-04-16T15:41:23.528304+0000","last_unstale":"2024-04-16T15:41:45.031572+0000","last_undegraded":"2024-04-16T15:41:45.031572+0000","last_fullsized":"2024-04-16T15:41:45.031572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:45.031483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:45.031483+0000","last_clean_scrub_stamp":"2024-04-16T15:41:45.031483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:18:22.498902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.279425+0000","last_change":"2024-04-16T15:41:36.711333+0000","last_active":"2024-04-16T15:41:39.279425+0000","last_peered":"2024-04-16T15:41:39.279425+0000","last_clean":"2024-04-16T15:41:39.279425+0000","last_became_active":"2024-04-16T15:41:23.519092+0000","last_became_peered":"2024-04-16T15:41:23.519092+0000","last_unstale":"2024-04-16T15:41:39.279425+0000","last_undegraded":"2024-04-16T15:41:39.279425+0000","last_fullsized":"2024-04-16T15:41:39.279425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:36.711217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:36.711217+0000","last_clean_scrub_stamp":"2024-04-16T15:41:36.711217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:59.520354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:49.078594+0000","last_change":"2024-04-16T15:41:49.078614+0000","last_active":"2024-04-16T15:41:49.078594+0000","last_peered":"2024-04-16T15:41:49.078594+0000","last_clean":"2024-04-16T15:41:49.078594+0000","last_became_active":"2024-04-16T15:41:23.535963+0000","last_became_peered":"2024-04-16T15:41:23.535963+0000","last_unstale":"2024-04-16T15:41:49.078594+0000","last_undegraded":"2024-04-16T15:41:49.078594+0000","last_fullsized":"2024-04-16T15:41:49.078594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:49.078488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:49.078488+0000","last_clean_scrub_stamp":"2024-04-16T15:41:49.078488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:06:12.121173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'5","reported_seq":32,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.281252+0000","last_change":"2024-04-16T15:41:35.946924+0000","last_active":"2024-04-16T15:41:39.281252+0000","last_peered":"2024-04-16T15:41:39.281252+0000","last_clean":"2024-04-16T15:41:39.281252+0000","last_became_active":"2024-04-16T15:41:22.522542+0000","last_became_peered":"2024-04-16T15:41:22.522542+0000","last_unstale":"2024-04-16T15:41:39.281252+0000","last_undegraded":"2024-04-16T15:41:39.281252+0000","last_fullsized":"2024-04-16T15:41:39.281252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'4","last_scrub_stamp":"2024-04-16T15:41:35.946791+0000","last_deep_scrub":"37'4","last_deep_scrub_stamp":"2024-04-16T15:41:35.946791+0000","last_clean_scrub_stamp":"2024-04-16T15:41:35.946791+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:59:11.572274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:41.712227+0000","last_change":"2024-04-16T15:41:41.712254+0000","last_active":"2024-04-16T15:41:41.712227+0000","last_peered":"2024-04-16T15:41:41.712227+0000","last_clean":"2024-04-16T15:41:41.712227+0000","last_became_active":"2024-04-16T15:41:23.535920+0000","last_became_peered":"2024-04-16T15:41:23.535920+0000","last_unstale":"2024-04-16T15:41:41.712227+0000","last_undegraded":"2024-04-16T15:41:41.712227+0000","last_fullsized":"2024-04-16T15:41:41.712227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:41.712141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:41.712141+0000","last_clean_scrub_stamp":"2024-04-16T15:41:41.712141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:48.386092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'2","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.282173+0000","last_change":"2024-04-16T15:41:34.837732+0000","last_active":"2024-04-16T15:41:39.282173+0000","last_peered":"2024-04-16T15:41:39.282173+0000","last_clean":"2024-04-16T15:41:39.282173+0000","last_became_active":"2024-04-16T15:41:22.523128+0000","last_became_peered":"2024-04-16T15:41:22.523128+0000","last_unstale":"2024-04-16T15:41:39.282173+0000","last_undegraded":"2024-04-16T15:41:39.282173+0000","last_fullsized":"2024-04-16T15:41:39.282173+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1","last_scrub_stamp":"2024-04-16T15:41:34.837636+0000","last_deep_scrub":"37'1","last_deep_scrub_stamp":"2024-04-16T15:41:34.837636+0000","last_clean_scrub_stamp":"2024-04-16T15:41:34.837636+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:31.876185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:43.305522+0000","last_change":"2024-04-16T15:41:43.305560+0000","last_active":"2024-04-16T15:41:43.305522+0000","last_peered":"2024-04-16T15:41:43.305522+0000","last_clean":"2024-04-16T15:41:43.305522+0000","last_became_active":"2024-04-16T15:41:23.530115+0000","last_became_peered":"2024-04-16T15:41:23.530115+0000","last_unstale":"2024-04-16T15:41:43.305522+0000","last_undegraded":"2024-04-16T15:41:43.305522+0000","last_fullsized":"2024-04-16T15:41:43.305522+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:43.305413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:43.305413+0000","last_clean_scrub_stamp":"2024-04-16T15:41:43.305413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:49.979364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.279305+0000","last_change":"2024-04-16T15:41:33.796229+0000","last_active":"2024-04-16T15:41:42.279305+0000","last_peered":"2024-04-16T15:41:42.279305+0000","last_clean":"2024-04-16T15:41:42.279305+0000","last_became_active":"2024-04-16T15:41:22.521643+0000","last_became_peered":"2024-04-16T15:41:22.521643+0000","last_unstale":"2024-04-16T15:41:42.279305+0000","last_undegraded":"2024-04-16T15:41:42.279305+0000","last_fullsized":"2024-04-16T15:41:42.279305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-04-16T15:41:33.796073+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-04-16T15:41:33.796073+0000","last_clean_scrub_stamp":"2024-04-16T15:41:33.796073+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:26.533282+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:45.747359+0000","last_change":"2024-04-16T15:41:45.747399+0000","last_active":"2024-04-16T15:41:45.747359+0000","last_peered":"2024-04-16T15:41:45.747359+0000","last_clean":"2024-04-16T15:41:45.747359+0000","last_became_active":"2024-04-16T15:41:23.528569+0000","last_became_peered":"2024-04-16T15:41:23.528569+0000","last_unstale":"2024-04-16T15:41:45.747359+0000","last_undegraded":"2024-04-16T15:41:45.747359+0000","last_fullsized":"2024-04-16T15:41:45.747359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:45.747259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:45.747259+0000","last_clean_scrub_stamp":"2024-04-16T15:41:45.747259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:57:08.560919+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.280507+0000","last_change":"2024-04-16T15:41:35.272400+0000","last_active":"2024-04-16T15:41:42.280507+0000","last_peered":"2024-04-16T15:41:42.280507+0000","last_clean":"2024-04-16T15:41:42.280507+0000","last_became_active":"2024-04-16T15:41:22.524864+0000","last_became_peered":"2024-04-16T15:41:22.524864+0000","last_unstale":"2024-04-16T15:41:42.280507+0000","last_undegraded":"2024-04-16T15:41:42.280507+0000","last_fullsized":"2024-04-16T15:41:42.280507+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-04-16T15:41:35.272283+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-04-16T15:41:35.272283+0000","last_clean_scrub_stamp":"2024-04-16T15:41:35.272283+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:28.009493+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.980595+0000","last_change":"2024-04-16T15:41:39.980621+0000","last_active":"2024-04-16T15:41:39.980595+0000","last_peered":"2024-04-16T15:41:39.980595+0000","last_clean":"2024-04-16T15:41:39.980595+0000","last_became_active":"2024-04-16T15:41:23.523587+0000","last_became_peered":"2024-04-16T15:41:23.523587+0000","last_unstale":"2024-04-16T15:41:39.980595+0000","last_undegraded":"2024-04-16T15:41:39.980595+0000","last_fullsized":"2024-04-16T15:41:39.980595+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:39.980518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:39.980518+0000","last_clean_scrub_stamp":"2024-04-16T15:41:39.980518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:24:04.357534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'2","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.275776+0000","last_change":"2024-04-16T15:41:38.001108+0000","last_active":"2024-04-16T15:41:39.275776+0000","last_peered":"2024-04-16T15:41:39.275776+0000","last_clean":"2024-04-16T15:41:39.275776+0000","last_became_active":"2024-04-16T15:41:22.525849+0000","last_became_peered":"2024-04-16T15:41:22.525849+0000","last_unstale":"2024-04-16T15:41:39.275776+0000","last_undegraded":"2024-04-16T15:41:39.275776+0000","last_fullsized":"2024-04-16T15:41:39.275776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1","last_scrub_stamp":"2024-04-16T15:41:38.000994+0000","last_deep_scrub":"37'1","last_deep_scrub_stamp":"2024-04-16T15:41:38.000994+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.000994+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:47:27.839545+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:43.077484+0000","last_change":"2024-04-16T15:41:43.077509+0000","last_active":"2024-04-16T15:41:43.077484+0000","last_peered":"2024-04-16T15:41:43.077484+0000","last_clean":"2024-04-16T15:41:43.077484+0000","last_became_active":"2024-04-16T15:41:23.520768+0000","last_became_peered":"2024-04-16T15:41:23.520768+0000","last_unstale":"2024-04-16T15:41:43.077484+0000","last_undegraded":"2024-04-16T15:41:43.077484+0000","last_fullsized":"2024-04-16T15:41:43.077484+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:43.077401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:43.077401+0000","last_clean_scrub_stamp":"2024-04-16T15:41:43.077401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:35.848374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'2","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.947502+0000","last_change":"2024-04-16T15:41:39.947550+0000","last_active":"2024-04-16T15:41:39.947502+0000","last_peered":"2024-04-16T15:41:39.947502+0000","last_clean":"2024-04-16T15:41:39.947502+0000","last_became_active":"2024-04-16T15:41:22.527999+0000","last_became_peered":"2024-04-16T15:41:22.527999+0000","last_unstale":"2024-04-16T15:41:39.947502+0000","last_undegraded":"2024-04-16T15:41:39.947502+0000","last_fullsized":"2024-04-16T15:41:39.947502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'2","last_scrub_stamp":"2024-04-16T15:41:39.947391+0000","last_deep_scrub":"39'2","last_deep_scrub_stamp":"2024-04-16T15:41:39.947391+0000","last_clean_scrub_stamp":"2024-04-16T15:41:39.947391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:52:26.514364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:44.317856+0000","last_change":"2024-04-16T15:41:44.317890+0000","last_active":"2024-04-16T15:41:44.317856+0000","last_peered":"2024-04-16T15:41:44.317856+0000","last_clean":"2024-04-16T15:41:44.317856+0000","last_became_active":"2024-04-16T15:41:23.535387+0000","last_became_peered":"2024-04-16T15:41:23.535387+0000","last_unstale":"2024-04-16T15:41:44.317856+0000","last_undegraded":"2024-04-16T15:41:44.317856+0000","last_fullsized":"2024-04-16T15:41:44.317856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:44.317771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:44.317771+0000","last_clean_scrub_stamp":"2024-04-16T15:41:44.317771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:17.637523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.279579+0000","last_change":"2024-04-16T15:41:34.760462+0000","last_active":"2024-04-16T15:41:39.279579+0000","last_peered":"2024-04-16T15:41:39.279579+0000","last_clean":"2024-04-16T15:41:39.279579+0000","last_became_active":"2024-04-16T15:41:22.525819+0000","last_became_peered":"2024-04-16T15:41:22.525819+0000","last_unstale":"2024-04-16T15:41:39.279579+0000","last_undegraded":"2024-04-16T15:41:39.279579+0000","last_fullsized":"2024-04-16T15:41:39.279579+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:34.760360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:34.760360+0000","last_clean_scrub_stamp":"2024-04-16T15:41:34.760360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:31.798909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:40.822678+0000","last_change":"2024-04-16T15:41:40.822713+0000","last_active":"2024-04-16T15:41:40.822678+0000","last_peered":"2024-04-16T15:41:40.822678+0000","last_clean":"2024-04-16T15:41:40.822678+0000","last_became_active":"2024-04-16T15:41:23.521900+0000","last_became_peered":"2024-04-16T15:41:23.521900+0000","last_unstale":"2024-04-16T15:41:40.822678+0000","last_undegraded":"2024-04-16T15:41:40.822678+0000","last_fullsized":"2024-04-16T15:41:40.822678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:40.822544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:40.822544+0000","last_clean_scrub_stamp":"2024-04-16T15:41:40.822544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:43.679693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.271376+0000","last_change":"2024-04-16T15:41:34.241134+0000","last_active":"2024-04-16T15:41:39.271376+0000","last_peered":"2024-04-16T15:41:39.271376+0000","last_clean":"2024-04-16T15:41:39.271376+0000","last_became_active":"2024-04-16T15:41:22.521234+0000","last_became_peered":"2024-04-16T15:41:22.521234+0000","last_unstale":"2024-04-16T15:41:39.271376+0000","last_undegraded":"2024-04-16T15:41:39.271376+0000","last_fullsized":"2024-04-16T15:41:39.271376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:34.241030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:34.241030+0000","last_clean_scrub_stamp":"2024-04-16T15:41:34.241030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:49.892312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:40.147497+0000","last_change":"2024-04-16T15:41:40.147534+0000","last_active":"2024-04-16T15:41:40.147497+0000","last_peered":"2024-04-16T15:41:40.147497+0000","last_clean":"2024-04-16T15:41:40.147497+0000","last_became_active":"2024-04-16T15:41:23.520819+0000","last_became_peered":"2024-04-16T15:41:23.520819+0000","last_unstale":"2024-04-16T15:41:40.147497+0000","last_undegraded":"2024-04-16T15:41:40.147497+0000","last_fullsized":"2024-04-16T15:41:40.147497+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:40.147384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:40.147384+0000","last_clean_scrub_stamp":"2024-04-16T15:41:40.147384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:59:15.772867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.271318+0000","last_change":"2024-04-16T15:41:33.801819+0000","last_active":"2024-04-16T15:41:39.271318+0000","last_peered":"2024-04-16T15:41:39.271318+0000","last_clean":"2024-04-16T15:41:39.271318+0000","last_became_active":"2024-04-16T15:41:22.518905+0000","last_became_peered":"2024-04-16T15:41:22.518905+0000","last_unstale":"2024-04-16T15:41:39.271318+0000","last_undegraded":"2024-04-16T15:41:39.271318+0000","last_fullsized":"2024-04-16T15:41:39.271318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:33.801680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:33.801680+0000","last_clean_scrub_stamp":"2024-04-16T15:41:33.801680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:16.715124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:46.014224+0000","last_change":"2024-04-16T15:41:46.014246+0000","last_active":"2024-04-16T15:41:46.014224+0000","last_peered":"2024-04-16T15:41:46.014224+0000","last_clean":"2024-04-16T15:41:46.014224+0000","last_became_active":"2024-04-16T15:41:23.526288+0000","last_became_peered":"2024-04-16T15:41:23.526288+0000","last_unstale":"2024-04-16T15:41:46.014224+0000","last_undegraded":"2024-04-16T15:41:46.014224+0000","last_fullsized":"2024-04-16T15:41:46.014224+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:46.014101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:46.014101+0000","last_clean_scrub_stamp":"2024-04-16T15:41:46.014101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:37.791182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.275946+0000","last_change":"2024-04-16T15:41:37.037745+0000","last_active":"2024-04-16T15:41:39.275946+0000","last_peered":"2024-04-16T15:41:39.275946+0000","last_clean":"2024-04-16T15:41:39.275946+0000","last_became_active":"2024-04-16T15:41:22.524567+0000","last_became_peered":"2024-04-16T15:41:22.524567+0000","last_unstale":"2024-04-16T15:41:39.275946+0000","last_undegraded":"2024-04-16T15:41:39.275946+0000","last_fullsized":"2024-04-16T15:41:39.275946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:37.037587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:37.037587+0000","last_clean_scrub_stamp":"2024-04-16T15:41:37.037587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:38:43.434220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.282068+0000","last_change":"2024-04-16T15:41:37.906256+0000","last_active":"2024-04-16T15:41:39.282068+0000","last_peered":"2024-04-16T15:41:39.282068+0000","last_clean":"2024-04-16T15:41:39.282068+0000","last_became_active":"2024-04-16T15:41:23.521984+0000","last_became_peered":"2024-04-16T15:41:23.521984+0000","last_unstale":"2024-04-16T15:41:39.282068+0000","last_undegraded":"2024-04-16T15:41:39.282068+0000","last_fullsized":"2024-04-16T15:41:39.282068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:37.906152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:37.906152+0000","last_clean_scrub_stamp":"2024-04-16T15:41:37.906152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:17:02.508112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":31,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.277967+0000","last_change":"2024-04-16T15:41:35.100694+0000","last_active":"2024-04-16T15:41:42.277967+0000","last_peered":"2024-04-16T15:41:42.277967+0000","last_clean":"2024-04-16T15:41:42.277967+0000","last_became_active":"2024-04-16T15:41:22.522759+0000","last_became_peered":"2024-04-16T15:41:22.522759+0000","last_unstale":"2024-04-16T15:41:42.277967+0000","last_undegraded":"2024-04-16T15:41:42.277967+0000","last_fullsized":"2024-04-16T15:41:42.277967+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3","last_scrub_stamp":"2024-04-16T15:41:35.100562+0000","last_deep_scrub":"37'3","last_deep_scrub_stamp":"2024-04-16T15:41:35.100562+0000","last_clean_scrub_stamp":"2024-04-16T15:41:35.100562+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:23:59.477578+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.330995+0000","last_change":"2024-04-16T15:41:42.331018+0000","last_active":"2024-04-16T15:41:42.330995+0000","last_peered":"2024-04-16T15:41:42.330995+0000","last_clean":"2024-04-16T15:41:42.330995+0000","last_became_active":"2024-04-16T15:41:23.529543+0000","last_became_peered":"2024-04-16T15:41:23.529543+0000","last_unstale":"2024-04-16T15:41:42.330995+0000","last_undegraded":"2024-04-16T15:41:42.330995+0000","last_fullsized":"2024-04-16T15:41:42.330995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:42.330891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:42.330891+0000","last_clean_scrub_stamp":"2024-04-16T15:41:42.330891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:52:28.897865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'7","reported_seq":38,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.279991+0000","last_change":"2024-04-16T15:41:36.820976+0000","last_active":"2024-04-16T15:41:42.279991+0000","last_peered":"2024-04-16T15:41:42.279991+0000","last_clean":"2024-04-16T15:41:42.279991+0000","last_became_active":"2024-04-16T15:41:22.522130+0000","last_became_peered":"2024-04-16T15:41:22.522130+0000","last_unstale":"2024-04-16T15:41:42.279991+0000","last_undegraded":"2024-04-16T15:41:42.279991+0000","last_fullsized":"2024-04-16T15:41:42.279991+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'6","last_scrub_stamp":"2024-04-16T15:41:36.820782+0000","last_deep_scrub":"37'6","last_deep_scrub_stamp":"2024-04-16T15:41:36.820782+0000","last_clean_scrub_stamp":"2024-04-16T15:41:36.820782+0000","objects_scrubbed":4,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:42:29.721343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.927995+0000","last_change":"2024-04-16T15:41:42.928033+0000","last_active":"2024-04-16T15:41:42.927995+0000","last_peered":"2024-04-16T15:41:42.927995+0000","last_clean":"2024-04-16T15:41:42.927995+0000","last_became_active":"2024-04-16T15:41:23.534825+0000","last_became_peered":"2024-04-16T15:41:23.534825+0000","last_unstale":"2024-04-16T15:41:42.927995+0000","last_undegraded":"2024-04-16T15:41:42.927995+0000","last_fullsized":"2024-04-16T15:41:42.927995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:42.927919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:42.927919+0000","last_clean_scrub_stamp":"2024-04-16T15:41:42.927919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:47:32.766469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.274606+0000","last_change":"2024-04-16T15:41:34.794950+0000","last_active":"2024-04-16T15:41:39.274606+0000","last_peered":"2024-04-16T15:41:39.274606+0000","last_clean":"2024-04-16T15:41:39.274606+0000","last_became_active":"2024-04-16T15:41:22.519829+0000","last_became_peered":"2024-04-16T15:41:22.519829+0000","last_unstale":"2024-04-16T15:41:39.274606+0000","last_undegraded":"2024-04-16T15:41:39.274606+0000","last_fullsized":"2024-04-16T15:41:39.274606+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-04-16T15:41:34.794858+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-04-16T15:41:34.794858+0000","last_clean_scrub_stamp":"2024-04-16T15:41:34.794858+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:52:21.361832+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:41.799268+0000","last_change":"2024-04-16T15:41:41.799291+0000","last_active":"2024-04-16T15:41:41.799268+0000","last_peered":"2024-04-16T15:41:41.799268+0000","last_clean":"2024-04-16T15:41:41.799268+0000","last_became_active":"2024-04-16T15:41:23.522159+0000","last_became_peered":"2024-04-16T15:41:23.522159+0000","last_unstale":"2024-04-16T15:41:41.799268+0000","last_undegraded":"2024-04-16T15:41:41.799268+0000","last_fullsized":"2024-04-16T15:41:41.799268+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:41.799186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:41.799186+0000","last_clean_scrub_stamp":"2024-04-16T15:41:41.799186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:34:41.694884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.279677+0000","last_change":"2024-04-16T15:41:35.721880+0000","last_active":"2024-04-16T15:41:39.279677+0000","last_peered":"2024-04-16T15:41:39.279677+0000","last_clean":"2024-04-16T15:41:39.279677+0000","last_became_active":"2024-04-16T15:41:22.525811+0000","last_became_peered":"2024-04-16T15:41:22.525811+0000","last_unstale":"2024-04-16T15:41:39.279677+0000","last_undegraded":"2024-04-16T15:41:39.279677+0000","last_fullsized":"2024-04-16T15:41:39.279677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:35.721778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:35.721778+0000","last_clean_scrub_stamp":"2024-04-16T15:41:35.721778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:15.590783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:43.983278+0000","last_change":"2024-04-16T15:41:43.983301+0000","last_active":"2024-04-16T15:41:43.983278+0000","last_peered":"2024-04-16T15:41:43.983278+0000","last_clean":"2024-04-16T15:41:43.983278+0000","last_became_active":"2024-04-16T15:41:23.527295+0000","last_became_peered":"2024-04-16T15:41:23.527295+0000","last_unstale":"2024-04-16T15:41:43.983278+0000","last_undegraded":"2024-04-16T15:41:43.983278+0000","last_fullsized":"2024-04-16T15:41:43.983278+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:43.983187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:43.983187+0000","last_clean_scrub_stamp":"2024-04-16T15:41:43.983187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:53:09.412908+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.271643+0000","last_change":"2024-04-16T15:41:36.973270+0000","last_active":"2024-04-16T15:41:39.271643+0000","last_peered":"2024-04-16T15:41:39.271643+0000","last_clean":"2024-04-16T15:41:39.271643+0000","last_became_active":"2024-04-16T15:41:22.523303+0000","last_became_peered":"2024-04-16T15:41:22.523303+0000","last_unstale":"2024-04-16T15:41:39.271643+0000","last_undegraded":"2024-04-16T15:41:39.271643+0000","last_fullsized":"2024-04-16T15:41:39.271643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:36.973162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:36.973162+0000","last_clean_scrub_stamp":"2024-04-16T15:41:36.973162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:17:11.577184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:43.794832+0000","last_change":"2024-04-16T15:41:43.794869+0000","last_active":"2024-04-16T15:41:43.794832+0000","last_peered":"2024-04-16T15:41:43.794832+0000","last_clean":"2024-04-16T15:41:43.794832+0000","last_became_active":"2024-04-16T15:41:23.522845+0000","last_became_peered":"2024-04-16T15:41:23.522845+0000","last_unstale":"2024-04-16T15:41:43.794832+0000","last_undegraded":"2024-04-16T15:41:43.794832+0000","last_fullsized":"2024-04-16T15:41:43.794832+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:43.794706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:43.794706+0000","last_clean_scrub_stamp":"2024-04-16T15:41:43.794706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:52:47.647531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":41,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.282698+0000","last_change":"2024-04-16T15:41:38.928016+0000","last_active":"2024-04-16T15:41:42.282698+0000","last_peered":"2024-04-16T15:41:42.282698+0000","last_clean":"2024-04-16T15:41:42.282698+0000","last_became_active":"2024-04-16T15:41:22.522633+0000","last_became_peered":"2024-04-16T15:41:22.522633+0000","last_unstale":"2024-04-16T15:41:42.282698+0000","last_undegraded":"2024-04-16T15:41:42.282698+0000","last_fullsized":"2024-04-16T15:41:42.282698+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'8","last_scrub_stamp":"2024-04-16T15:41:38.927889+0000","last_deep_scrub":"37'8","last_deep_scrub_stamp":"2024-04-16T15:41:38.927889+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.927889+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:21.841333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:44.768977+0000","last_change":"2024-04-16T15:41:44.769017+0000","last_active":"2024-04-16T15:41:44.768977+0000","last_peered":"2024-04-16T15:41:44.768977+0000","last_clean":"2024-04-16T15:41:44.768977+0000","last_became_active":"2024-04-16T15:41:23.527841+0000","last_became_peered":"2024-04-16T15:41:23.527841+0000","last_unstale":"2024-04-16T15:41:44.768977+0000","last_undegraded":"2024-04-16T15:41:44.768977+0000","last_fullsized":"2024-04-16T15:41:44.768977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:44.768877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:44.768877+0000","last_clean_scrub_stamp":"2024-04-16T15:41:44.768877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:50.628081+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.276000+0000","last_change":"2024-04-16T15:41:38.993108+0000","last_active":"2024-04-16T15:41:39.276000+0000","last_peered":"2024-04-16T15:41:39.276000+0000","last_clean":"2024-04-16T15:41:39.276000+0000","last_became_active":"2024-04-16T15:41:22.526140+0000","last_became_peered":"2024-04-16T15:41:22.526140+0000","last_unstale":"2024-04-16T15:41:39.276000+0000","last_undegraded":"2024-04-16T15:41:39.276000+0000","last_fullsized":"2024-04-16T15:41:39.276000+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1","last_scrub_stamp":"2024-04-16T15:41:38.992989+0000","last_deep_scrub":"37'1","last_deep_scrub_stamp":"2024-04-16T15:41:38.992989+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.992989+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:34:07.986894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.080153+0000","last_change":"2024-04-16T15:41:42.080196+0000","last_active":"2024-04-16T15:41:42.080153+0000","last_peered":"2024-04-16T15:41:42.080153+0000","last_clean":"2024-04-16T15:41:42.080153+0000","last_became_active":"2024-04-16T15:41:23.521041+0000","last_became_peered":"2024-04-16T15:41:23.521041+0000","last_unstale":"2024-04-16T15:41:42.080153+0000","last_undegraded":"2024-04-16T15:41:42.080153+0000","last_fullsized":"2024-04-16T15:41:42.080153+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:42.080073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:42.080073+0000","last_clean_scrub_stamp":"2024-04-16T15:41:42.080073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:19:33.857153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.280979+0000","last_change":"2024-04-16T15:41:37.238793+0000","last_active":"2024-04-16T15:41:42.280979+0000","last_peered":"2024-04-16T15:41:42.280979+0000","last_clean":"2024-04-16T15:41:42.280979+0000","last_became_active":"2024-04-16T15:41:22.528745+0000","last_became_peered":"2024-04-16T15:41:22.528745+0000","last_unstale":"2024-04-16T15:41:42.280979+0000","last_undegraded":"2024-04-16T15:41:42.280979+0000","last_fullsized":"2024-04-16T15:41:42.280979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-04-16T15:41:37.238672+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-04-16T15:41:37.238672+0000","last_clean_scrub_stamp":"2024-04-16T15:41:37.238672+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:17.107678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.278887+0000","last_change":"2024-04-16T15:41:39.278921+0000","last_active":"2024-04-16T15:41:39.278887+0000","last_peered":"2024-04-16T15:41:39.278887+0000","last_clean":"2024-04-16T15:41:39.278887+0000","last_became_active":"2024-04-16T15:41:23.521964+0000","last_became_peered":"2024-04-16T15:41:23.521964+0000","last_unstale":"2024-04-16T15:41:39.278887+0000","last_undegraded":"2024-04-16T15:41:39.278887+0000","last_fullsized":"2024-04-16T15:41:39.278887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:39.278772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:39.278772+0000","last_clean_scrub_stamp":"2024-04-16T15:41:39.278772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:17:03.880733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.276204+0000","last_change":"2024-04-16T15:41:34.092498+0000","last_active":"2024-04-16T15:41:39.276204+0000","last_peered":"2024-04-16T15:41:39.276204+0000","last_clean":"2024-04-16T15:41:39.276204+0000","last_became_active":"2024-04-16T15:41:22.522888+0000","last_became_peered":"2024-04-16T15:41:22.522888+0000","last_unstale":"2024-04-16T15:41:39.276204+0000","last_undegraded":"2024-04-16T15:41:39.276204+0000","last_fullsized":"2024-04-16T15:41:39.276204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1","last_scrub_stamp":"2024-04-16T15:41:34.092381+0000","last_deep_scrub":"37'1","last_deep_scrub_stamp":"2024-04-16T15:41:34.092381+0000","last_clean_scrub_stamp":"2024-04-16T15:41:34.092381+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:41:40.542061+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:44.121983+0000","last_change":"2024-04-16T15:41:44.122005+0000","last_active":"2024-04-16T15:41:44.121983+0000","last_peered":"2024-04-16T15:41:44.121983+0000","last_clean":"2024-04-16T15:41:44.121983+0000","last_became_active":"2024-04-16T15:41:23.525537+0000","last_became_peered":"2024-04-16T15:41:23.525537+0000","last_unstale":"2024-04-16T15:41:44.121983+0000","last_undegraded":"2024-04-16T15:41:44.121983+0000","last_fullsized":"2024-04-16T15:41:44.121983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:44.121902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:44.121902+0000","last_clean_scrub_stamp":"2024-04-16T15:41:44.121902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:13:07.268869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.279638+0000","last_change":"2024-04-16T15:41:32.759944+0000","last_active":"2024-04-16T15:41:39.279638+0000","last_peered":"2024-04-16T15:41:39.279638+0000","last_clean":"2024-04-16T15:41:39.279638+0000","last_became_active":"2024-04-16T15:41:22.521574+0000","last_became_peered":"2024-04-16T15:41:22.521574+0000","last_unstale":"2024-04-16T15:41:39.279638+0000","last_undegraded":"2024-04-16T15:41:39.279638+0000","last_fullsized":"2024-04-16T15:41:39.279638+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:32.759848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:32.759848+0000","last_clean_scrub_stamp":"2024-04-16T15:41:32.759848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:48.411129+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:46.998612+0000","last_change":"2024-04-16T15:41:46.998639+0000","last_active":"2024-04-16T15:41:46.998612+0000","last_peered":"2024-04-16T15:41:46.998612+0000","last_clean":"2024-04-16T15:41:46.998612+0000","last_became_active":"2024-04-16T15:41:23.528946+0000","last_became_peered":"2024-04-16T15:41:23.528946+0000","last_unstale":"2024-04-16T15:41:46.998612+0000","last_undegraded":"2024-04-16T15:41:46.998612+0000","last_fullsized":"2024-04-16T15:41:46.998612+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:46.998527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:46.998527+0000","last_clean_scrub_stamp":"2024-04-16T15:41:46.998527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:39.769500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'3","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.276291+0000","last_change":"2024-04-16T15:41:37.112225+0000","last_active":"2024-04-16T15:41:39.276291+0000","last_peered":"2024-04-16T15:41:39.276291+0000","last_clean":"2024-04-16T15:41:39.276291+0000","last_became_active":"2024-04-16T15:41:22.525228+0000","last_became_peered":"2024-04-16T15:41:22.525228+0000","last_unstale":"2024-04-16T15:41:39.276291+0000","last_undegraded":"2024-04-16T15:41:39.276291+0000","last_fullsized":"2024-04-16T15:41:39.276291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-04-16T15:41:37.112063+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-04-16T15:41:37.112063+0000","last_clean_scrub_stamp":"2024-04-16T15:41:37.112063+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:26:31.459835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.281833+0000","last_change":"2024-04-16T15:41:36.884236+0000","last_active":"2024-04-16T15:41:39.281833+0000","last_peered":"2024-04-16T15:41:39.281833+0000","last_clean":"2024-04-16T15:41:39.281833+0000","last_became_active":"2024-04-16T15:41:23.521931+0000","last_became_peered":"2024-04-16T15:41:23.521931+0000","last_unstale":"2024-04-16T15:41:39.281833+0000","last_undegraded":"2024-04-16T15:41:39.281833+0000","last_fullsized":"2024-04-16T15:41:39.281833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:36.884110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:36.884110+0000","last_clean_scrub_stamp":"2024-04-16T15:41:36.884110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:05:00.415368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.275925+0000","last_change":"2024-04-16T15:41:35.122705+0000","last_active":"2024-04-16T15:41:39.275925+0000","last_peered":"2024-04-16T15:41:39.275925+0000","last_clean":"2024-04-16T15:41:39.275925+0000","last_became_active":"2024-04-16T15:41:22.523348+0000","last_became_peered":"2024-04-16T15:41:22.523348+0000","last_unstale":"2024-04-16T15:41:39.275925+0000","last_undegraded":"2024-04-16T15:41:39.275925+0000","last_fullsized":"2024-04-16T15:41:39.275925+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:35.122584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:35.122584+0000","last_clean_scrub_stamp":"2024-04-16T15:41:35.122584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:09.901431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:47.086553+0000","last_change":"2024-04-16T15:41:47.086576+0000","last_active":"2024-04-16T15:41:47.086553+0000","last_peered":"2024-04-16T15:41:47.086553+0000","last_clean":"2024-04-16T15:41:47.086553+0000","last_became_active":"2024-04-16T15:41:23.524540+0000","last_became_peered":"2024-04-16T15:41:23.524540+0000","last_unstale":"2024-04-16T15:41:47.086553+0000","last_undegraded":"2024-04-16T15:41:47.086553+0000","last_fullsized":"2024-04-16T15:41:47.086553+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:47.086471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:47.086471+0000","last_clean_scrub_stamp":"2024-04-16T15:41:47.086471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:21:38.848490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.282110+0000","last_change":"2024-04-16T15:41:36.902767+0000","last_active":"2024-04-16T15:41:39.282110+0000","last_peered":"2024-04-16T15:41:39.282110+0000","last_clean":"2024-04-16T15:41:39.282110+0000","last_became_active":"2024-04-16T15:41:22.521860+0000","last_became_peered":"2024-04-16T15:41:22.521860+0000","last_unstale":"2024-04-16T15:41:39.282110+0000","last_undegraded":"2024-04-16T15:41:39.282110+0000","last_fullsized":"2024-04-16T15:41:39.282110+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:36.902664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:36.902664+0000","last_clean_scrub_stamp":"2024-04-16T15:41:36.902664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:11:29.162019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.804673+0000","last_change":"2024-04-16T15:41:39.804695+0000","last_active":"2024-04-16T15:41:39.804673+0000","last_peered":"2024-04-16T15:41:39.804673+0000","last_clean":"2024-04-16T15:41:39.804673+0000","last_became_active":"2024-04-16T15:41:23.521852+0000","last_became_peered":"2024-04-16T15:41:23.521852+0000","last_unstale":"2024-04-16T15:41:39.804673+0000","last_undegraded":"2024-04-16T15:41:39.804673+0000","last_fullsized":"2024-04-16T15:41:39.804673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:39.804591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:39.804591+0000","last_clean_scrub_stamp":"2024-04-16T15:41:39.804591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:39.838354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.275819+0000","last_change":"2024-04-16T15:41:38.063694+0000","last_active":"2024-04-16T15:41:39.275819+0000","last_peered":"2024-04-16T15:41:39.275819+0000","last_clean":"2024-04-16T15:41:39.275819+0000","last_became_active":"2024-04-16T15:41:22.525423+0000","last_became_peered":"2024-04-16T15:41:22.525423+0000","last_unstale":"2024-04-16T15:41:39.275819+0000","last_undegraded":"2024-04-16T15:41:39.275819+0000","last_fullsized":"2024-04-16T15:41:39.275819+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:38.063567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:38.063567+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.063567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:24.567495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:40.691321+0000","last_change":"2024-04-16T15:41:40.691346+0000","last_active":"2024-04-16T15:41:40.691321+0000","last_peered":"2024-04-16T15:41:40.691321+0000","last_clean":"2024-04-16T15:41:40.691321+0000","last_became_active":"2024-04-16T15:41:23.535863+0000","last_became_peered":"2024-04-16T15:41:23.535863+0000","last_unstale":"2024-04-16T15:41:40.691321+0000","last_undegraded":"2024-04-16T15:41:40.691321+0000","last_fullsized":"2024-04-16T15:41:40.691321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:40.691232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:40.691232+0000","last_clean_scrub_stamp":"2024-04-16T15:41:40.691232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:52:27.258206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.281457+0000","last_change":"2024-04-16T15:41:36.078650+0000","last_active":"2024-04-16T15:41:42.281457+0000","last_peered":"2024-04-16T15:41:42.281457+0000","last_clean":"2024-04-16T15:41:42.281457+0000","last_became_active":"2024-04-16T15:41:22.524464+0000","last_became_peered":"2024-04-16T15:41:22.524464+0000","last_unstale":"2024-04-16T15:41:42.281457+0000","last_undegraded":"2024-04-16T15:41:42.281457+0000","last_fullsized":"2024-04-16T15:41:42.281457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-04-16T15:41:36.078506+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-04-16T15:41:36.078506+0000","last_clean_scrub_stamp":"2024-04-16T15:41:36.078506+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:52:15.050487+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:42.012905+0000","last_change":"2024-04-16T15:41:42.012931+0000","last_active":"2024-04-16T15:41:42.012905+0000","last_peered":"2024-04-16T15:41:42.012905+0000","last_clean":"2024-04-16T15:41:42.012905+0000","last_became_active":"2024-04-16T15:41:23.526580+0000","last_became_peered":"2024-04-16T15:41:23.526580+0000","last_unstale":"2024-04-16T15:41:42.012905+0000","last_undegraded":"2024-04-16T15:41:42.012905+0000","last_fullsized":"2024-04-16T15:41:42.012905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:42.012794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:42.012794+0000","last_clean_scrub_stamp":"2024-04-16T15:41:42.012794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:47:31.851344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":26,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.276222+0000","last_change":"2024-04-16T15:41:36.094613+0000","last_active":"2024-04-16T15:41:39.276222+0000","last_peered":"2024-04-16T15:41:39.276222+0000","last_clean":"2024-04-16T15:41:39.276222+0000","last_became_active":"2024-04-16T15:41:22.523524+0000","last_became_peered":"2024-04-16T15:41:22.523524+0000","last_unstale":"2024-04-16T15:41:39.276222+0000","last_undegraded":"2024-04-16T15:41:39.276222+0000","last_fullsized":"2024-04-16T15:41:39.276222+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:36.094527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:36.094527+0000","last_clean_scrub_stamp":"2024-04-16T15:41:36.094527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:32.153064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.270999+0000","last_change":"2024-04-16T15:41:38.830715+0000","last_active":"2024-04-16T15:41:39.270999+0000","last_peered":"2024-04-16T15:41:39.270999+0000","last_clean":"2024-04-16T15:41:39.270999+0000","last_became_active":"2024-04-16T15:41:23.521114+0000","last_became_peered":"2024-04-16T15:41:23.521114+0000","last_unstale":"2024-04-16T15:41:39.270999+0000","last_undegraded":"2024-04-16T15:41:39.270999+0000","last_fullsized":"2024-04-16T15:41:39.270999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:38.830577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:38.830577+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.830577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:14.956356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.282070+0000","last_change":"2024-04-16T15:41:32.919303+0000","last_active":"2024-04-16T15:41:39.282070+0000","last_peered":"2024-04-16T15:41:39.282070+0000","last_clean":"2024-04-16T15:41:39.282070+0000","last_became_active":"2024-04-16T15:41:22.516776+0000","last_became_peered":"2024-04-16T15:41:22.516776+0000","last_unstale":"2024-04-16T15:41:39.282070+0000","last_undegraded":"2024-04-16T15:41:39.282070+0000","last_fullsized":"2024-04-16T15:41:39.282070+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:32.919207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:32.919207+0000","last_clean_scrub_stamp":"2024-04-16T15:41:32.919207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:15:48.570489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.281928+0000","last_change":"2024-04-16T15:41:35.843266+0000","last_active":"2024-04-16T15:41:39.281928+0000","last_peered":"2024-04-16T15:41:39.281928+0000","last_clean":"2024-04-16T15:41:39.281928+0000","last_became_active":"2024-04-16T15:41:23.521351+0000","last_became_peered":"2024-04-16T15:41:23.521351+0000","last_unstale":"2024-04-16T15:41:39.281928+0000","last_undegraded":"2024-04-16T15:41:39.281928+0000","last_fullsized":"2024-04-16T15:41:39.281928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:35.843156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:35.843156+0000","last_clean_scrub_stamp":"2024-04-16T15:41:35.843156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:40:15.712162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.282045+0000","last_change":"2024-04-16T15:41:33.877420+0000","last_active":"2024-04-16T15:41:39.282045+0000","last_peered":"2024-04-16T15:41:39.282045+0000","last_clean":"2024-04-16T15:41:39.282045+0000","last_became_active":"2024-04-16T15:41:22.517494+0000","last_became_peered":"2024-04-16T15:41:22.517494+0000","last_unstale":"2024-04-16T15:41:39.282045+0000","last_undegraded":"2024-04-16T15:41:39.282045+0000","last_fullsized":"2024-04-16T15:41:39.282045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:33.877326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:33.877326+0000","last_clean_scrub_stamp":"2024-04-16T15:41:33.877326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:43:26.614536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.279580+0000","last_change":"2024-04-16T15:41:38.659647+0000","last_active":"2024-04-16T15:41:39.279580+0000","last_peered":"2024-04-16T15:41:39.279580+0000","last_clean":"2024-04-16T15:41:39.279580+0000","last_became_active":"2024-04-16T15:41:23.523821+0000","last_became_peered":"2024-04-16T15:41:23.523821+0000","last_unstale":"2024-04-16T15:41:39.279580+0000","last_undegraded":"2024-04-16T15:41:39.279580+0000","last_fullsized":"2024-04-16T15:41:39.279580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:38.659523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:38.659523+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.659523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:44:28.257712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.272523+0000","last_change":"2024-04-16T15:41:34.990684+0000","last_active":"2024-04-16T15:41:39.272523+0000","last_peered":"2024-04-16T15:41:39.272523+0000","last_clean":"2024-04-16T15:41:39.272523+0000","last_became_active":"2024-04-16T15:41:22.518999+0000","last_became_peered":"2024-04-16T15:41:22.518999+0000","last_unstale":"2024-04-16T15:41:39.272523+0000","last_undegraded":"2024-04-16T15:41:39.272523+0000","last_fullsized":"2024-04-16T15:41:39.272523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:34.990544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:34.990544+0000","last_clean_scrub_stamp":"2024-04-16T15:41:34.990544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:59:38.422964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.271378+0000","last_change":"2024-04-16T15:41:38.278758+0000","last_active":"2024-04-16T15:41:39.271378+0000","last_peered":"2024-04-16T15:41:39.271378+0000","last_clean":"2024-04-16T15:41:39.271378+0000","last_became_active":"2024-04-16T15:41:23.522254+0000","last_became_peered":"2024-04-16T15:41:23.522254+0000","last_unstale":"2024-04-16T15:41:39.271378+0000","last_undegraded":"2024-04-16T15:41:39.271378+0000","last_fullsized":"2024-04-16T15:41:39.271378+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:38.278642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:38.278642+0000","last_clean_scrub_stamp":"2024-04-16T15:41:38.278642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:33:01.087779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.271341+0000","last_change":"2024-04-16T15:41:36.227310+0000","last_active":"2024-04-16T15:41:39.271341+0000","last_peered":"2024-04-16T15:41:39.271341+0000","last_clean":"2024-04-16T15:41:39.271341+0000","last_became_active":"2024-04-16T15:41:22.523261+0000","last_became_peered":"2024-04-16T15:41:22.523261+0000","last_unstale":"2024-04-16T15:41:39.271341+0000","last_undegraded":"2024-04-16T15:41:39.271341+0000","last_fullsized":"2024-04-16T15:41:39.271341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:36.227180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:36.227180+0000","last_clean_scrub_stamp":"2024-04-16T15:41:36.227180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:18:33.265729+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:40.995295+0000","last_change":"2024-04-16T15:41:40.995333+0000","last_active":"2024-04-16T15:41:40.995295+0000","last_peered":"2024-04-16T15:41:40.995295+0000","last_clean":"2024-04-16T15:41:40.995295+0000","last_became_active":"2024-04-16T15:41:23.526499+0000","last_became_peered":"2024-04-16T15:41:23.526499+0000","last_unstale":"2024-04-16T15:41:40.995295+0000","last_undegraded":"2024-04-16T15:41:40.995295+0000","last_fullsized":"2024-04-16T15:41:40.995295+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:40.995203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:40.995203+0000","last_clean_scrub_stamp":"2024-04-16T15:41:40.995203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T16:10:47.669153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.270673+0000","last_change":"2024-04-16T15:41:35.802834+0000","last_active":"2024-04-16T15:41:39.270673+0000","last_peered":"2024-04-16T15:41:39.270673+0000","last_clean":"2024-04-16T15:41:39.270673+0000","last_became_active":"2024-04-16T15:41:22.520944+0000","last_became_peered":"2024-04-16T15:41:22.520944+0000","last_unstale":"2024-04-16T15:41:39.270673+0000","last_undegraded":"2024-04-16T15:41:39.270673+0000","last_fullsized":"2024-04-16T15:41:39.270673+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:35.802744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:35.802744+0000","last_clean_scrub_stamp":"2024-04-16T15:41:35.802744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:37:09.122496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":23,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:50.097921+0000","last_change":"2024-04-16T15:41:50.097942+0000","last_active":"2024-04-16T15:41:50.097921+0000","last_peered":"2024-04-16T15:41:50.097921+0000","last_clean":"2024-04-16T15:41:50.097921+0000","last_became_active":"2024-04-16T15:41:23.530755+0000","last_became_peered":"2024-04-16T15:41:23.530755+0000","last_unstale":"2024-04-16T15:41:50.097921+0000","last_undegraded":"2024-04-16T15:41:50.097921+0000","last_fullsized":"2024-04-16T15:41:50.097921+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:50.097843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:50.097843+0000","last_clean_scrub_stamp":"2024-04-16T15:41:50.097843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:59.478456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.282204+0000","last_change":"2024-04-16T15:41:37.930028+0000","last_active":"2024-04-16T15:41:39.282204+0000","last_peered":"2024-04-16T15:41:39.282204+0000","last_clean":"2024-04-16T15:41:39.282204+0000","last_became_active":"2024-04-16T15:41:22.522766+0000","last_became_peered":"2024-04-16T15:41:22.522766+0000","last_unstale":"2024-04-16T15:41:39.282204+0000","last_undegraded":"2024-04-16T15:41:39.282204+0000","last_fullsized":"2024-04-16T15:41:39.282204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T15:41:37.929893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T15:41:37.929893+0000","last_clean_scrub_stamp":"2024-04-16T15:41:37.929893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T15:43:39.936272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"38'119","reported_seq":215,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-16T15:41:39.282093+0000","last_change":"2024-04-16T15:41:35.890851+0000","last_active":"2024-04-16T15:41:39.282093+0000","last_peered":"2024-04-16T15:41:39.282093+0000","last_clean":"2024-04-16T15:41:39.282093+0000","last_became_active":"2024-04-16T15:40:12.956835+0000","last_became_peered":"2024-04-16T15:40:12.956835+0000","last_unstale":"2024-04-16T15:41:39.282093+0000","last_undegraded":"2024-04-16T15:41:39.282093+0000","last_fullsized":"2024-04-16T15:41:39.282093+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"38'119","last_scrub_stamp":"2024-04-16T15:41:35.890737+0000","last_deep_scrub":"38'119","last_deep_scrub_stamp":"2024-04-16T15:41:35.890737+0000","last_clean_scrub_stamp":"2024-04-16T15:41:35.890737+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:58.699874+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":77,"num_read_kb":80,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1060864,"data_stored":788721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":454656,"data_stored":197253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":458752,"data_stored":196783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":458752,"data_stored":197275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":466944,"data_stored":198317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":454656,"data_stored":196741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":196761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1048576,"data_stored":787073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T15:41:56.065 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-16T15:41:56.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.203+0000 7f54da782640 1 Processor -- start 2024-04-16T15:41:56.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.203+0000 7f54da782640 1 -- start start 2024-04-16T15:41:56.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54da782640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f54d406e0a0 0x7f54d4154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54da782640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d40c3e20 con 0x7f54d40ac9a0 2024-04-16T15:41:56.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54da782640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d40a9e90 con 0x7f54d40ac5c0 2024-04-16T15:41:56.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54da782640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f54d40ace30 con 0x7f54d406e0a0 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54d8d1f640 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f54d40ac9a0 0x7f54d415e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:44202/0 (socket says 172.21.15.71:44202) 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54d8d1f640 1 -- 172.21.15.71:0/1539474896 learned_addr learned my addr 172.21.15.71:0/1539474896 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54d37fe640 1 --2- 172.21.15.71:0/1539474896 >> v2:172.21.15.122:3300/0 conn(0x7f54d406e0a0 0x7f54d4154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54d2ffd640 1 -- 172.21.15.71:0/1539474896 <== mon.0 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713646182 0 0) 0x7f54d40c3e20 con 0x7f54d40ac9a0 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.204+0000 7f54d2ffd640 1 -- 172.21.15.71:0/1539474896 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54bc003520 con 0x7f54d40ac9a0 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.205+0000 7f54d37fe640 1 -- 172.21.15.71:0/1539474896 >> v1:172.21.15.122:6789/0 conn(0x7f54d40ac5c0 legacy=0x7f54d40c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.205+0000 7f54d2ffd640 1 -- 172.21.15.71:0/1539474896 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252743310 0 0) 0x7f54d40a9e90 con 0x7f54d40ac5c0 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.205+0000 7f54d37fe640 1 -- 172.21.15.71:0/1539474896 >> v1:172.21.15.71:6789/0 conn(0x7f54d40ac9a0 legacy=0x7f54d415e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.205+0000 7f54d37fe640 1 -- 172.21.15.71:0/1539474896 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d40aa170 con 0x7f54d406e0a0 2024-04-16T15:41:56.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.205+0000 7f54d37fe640 1 --2- 172.21.15.71:0/1539474896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d406e0a0 0x7f54d4154830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54c80097a0 tx=0x7f54c802b740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdfadba348eaf7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:56.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54d2ffd640 1 -- 172.21.15.71:0/1539474896 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8037070 con 0x7f54d406e0a0 2024-04-16T15:41:56.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54d2ffd640 1 -- 172.21.15.71:0/1539474896 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c8030510 con 0x7f54d406e0a0 2024-04-16T15:41:56.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54d2ffd640 1 -- 172.21.15.71:0/1539474896 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc001970 con 0x7f54d406e0a0 2024-04-16T15:41:56.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54da782640 1 -- 172.21.15.71:0/1539474896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d406e0a0 msgr2=0x7f54d4154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54da782640 1 --2- 172.21.15.71:0/1539474896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d406e0a0 0x7f54d4154830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54c80097a0 tx=0x7f54c802b740 comp rx=0 tx=0).stop 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54da782640 1 -- 172.21.15.71:0/1539474896 shutdown_connections 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54da782640 1 --2- 172.21.15.71:0/1539474896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d406e0a0 0x7f54d4154830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54da782640 1 -- 172.21.15.71:0/1539474896 >> 172.21.15.71:0/1539474896 conn(0x7f54d40c8470 msgr2=0x7f54d40c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54da782640 1 -- 172.21.15.71:0/1539474896 shutdown_connections 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.206+0000 7f54da782640 1 -- 172.21.15.71:0/1539474896 wait complete. 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.207+0000 7f54da782640 1 Processor -- start 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.207+0000 7f54da782640 1 -- start start 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54da782640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54d406e0a0 0x7f54d414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54da782640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d40ac5c0 0x7f54d414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54da782640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40ac9a0 0x7f54d41eed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54da782640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4154030 con 0x7f54d40ac9a0 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54da782640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41543a0 con 0x7f54d40ac5c0 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54da782640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f54d4153d00 con 0x7f54d406e0a0 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d8d1f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40ac9a0 0x7f54d41eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d8d1f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40ac9a0 0x7f54d41eed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42720/0 (socket says 172.21.15.71:42720) 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d8d1f640 1 -- 172.21.15.71:0/627139586 learned_addr learned my addr 172.21.15.71:0/627139586 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d3fff640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54d406e0a0 0x7f54d414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d37fe640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d40ac5c0 0x7f54d414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d8d1f640 1 -- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54d406e0a0 msgr2=0x7f54d414cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d8d1f640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54d406e0a0 0x7f54d414cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d8d1f640 1 -- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d40ac5c0 msgr2=0x7f54d414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d8d1f640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d40ac5c0 0x7f54d414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d8d1f640 1 -- 172.21.15.71:0/627139586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d4157e10 con 0x7f54d40ac9a0 2024-04-16T15:41:56.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d3fff640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54d406e0a0 0x7f54d414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:56.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.208+0000 7f54d37fe640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d40ac5c0 0x7f54d414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T15:41:56.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.209+0000 7f54d8d1f640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40ac9a0 0x7f54d41eed70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00caf0 tx=0x7f54cc002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.209+0000 7f54d0ff9640 1 -- 172.21.15.71:0/627139586 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc018070 con 0x7f54d40ac9a0 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.209+0000 7f54d0ff9640 1 -- 172.21.15.71:0/627139586 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc00a7c0 con 0x7f54d40ac9a0 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.209+0000 7f54d0ff9640 1 -- 172.21.15.71:0/627139586 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00ab90 con 0x7f54d40ac9a0 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.209+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d41580f0 con 0x7f54d40ac9a0 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.209+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d406d890 con 0x7f54d40ac9a0 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.210+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d40a9ec0 con 0x7f54d40ac9a0 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.210+0000 7f54d0ff9640 1 -- 172.21.15.71:0/627139586 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54cc01b070 con 0x7f54d40ac9a0 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.211+0000 7f54d0ff9640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f54c006c920 0x7f54c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.211+0000 7f54d0ff9640 1 -- 172.21.15.71:0/627139586 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7f54cc08b5e0 con 0x7f54d40ac9a0 2024-04-16T15:41:56.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.211+0000 7f54d3fff640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f54c006c920 0x7f54c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.211+0000 7f54d3fff640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f54c006c920 0x7f54c006ede0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f54c4001120 tx=0x7f54c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:56.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.215+0000 7f54d0ff9640 1 -- 172.21.15.71:0/627139586 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54d40a9ec0 con 0x7f54d40ac9a0 2024-04-16T15:41:56.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.390+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f54d4068be0 con 0x7f54d40ac9a0 2024-04-16T15:41:56.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.393+0000 7f54d0ff9640 1 -- 172.21.15.71:0/627139586 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f54cc059830 con 0x7f54d40ac9a0 2024-04-16T15:41:56.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.394+0000 7f54d0ff9640 1 -- 172.21.15.71:0/627139586 <== mon.0 v2:172.21.15.71:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f54cc059130 con 0x7f54d40ac9a0 2024-04-16T15:41:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f54c006c920 msgr2=0x7f54c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f54c006c920 0x7f54c006ede0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f54c4001120 tx=0x7f54c4009040 comp rx=0 tx=0).stop 2024-04-16T15:41:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40ac9a0 msgr2=0x7f54d41eed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40ac9a0 0x7f54d41eed70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00caf0 tx=0x7f54cc002190 comp rx=0 tx=0).stop 2024-04-16T15:41:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 shutdown_connections 2024-04-16T15:41:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7f54c006c920 0x7f54c006ede0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40ac9a0 0x7f54d41eed70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54d40ac5c0 0x7f54d414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 --2- 172.21.15.71:0/627139586 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f54d406e0a0 0x7f54d414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 >> 172.21.15.71:0/627139586 conn(0x7f54d40c8470 msgr2=0x7f54d40c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 shutdown_connections 2024-04-16T15:41:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.396+0000 7f54da782640 1 -- 172.21.15.71:0/627139586 wait complete. 2024-04-16T15:41:56.406 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-16T15:41:56.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T15:41:56.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.511+0000 7efd2f88f640 1 Processor -- start 2024-04-16T15:41:56.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.511+0000 7efd2f88f640 1 -- start start 2024-04-16T15:41:56.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2f88f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7efd2806cbb0 0x7efd28072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2f88f640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7efd280c3170 0x7efd2815f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2f88f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd280c5b00 con 0x7efd280c3f40 2024-04-16T15:41:56.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2f88f640 1 -- --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 -- 0x7efd280c6390 con 0x7efd280c3170 2024-04-16T15:41:56.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2f88f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7efd280c6670 con 0x7efd2806cbb0 2024-04-16T15:41:56.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2de2c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7efd280c3170 0x7efd2815f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2de2c640 1 --2- >> v2:172.21.15.71:3300/0 conn(0x7efd280c3170 0x7efd2815f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42732/0 (socket says 172.21.15.71:42732) 2024-04-16T15:41:56.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2de2c640 1 -- 172.21.15.71:0/2797508342 learned_addr learned my addr 172.21.15.71:0/2797508342 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:56.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2d62b640 1 --2- 172.21.15.71:0/2797508342 >> v2:172.21.15.122:3300/0 conn(0x7efd2806cbb0 0x7efd28072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.512+0000 7efd2de2c640 1 -- 172.21.15.71:0/2797508342 >> v2:172.21.15.122:3300/0 conn(0x7efd2806cbb0 msgr2=0x7efd28072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T15:41:56.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.513+0000 7efd2de2c640 1 --2- 172.21.15.71:0/2797508342 >> v2:172.21.15.122:3300/0 conn(0x7efd2806cbb0 0x7efd28072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.513+0000 7efd2de2c640 1 -- 172.21.15.71:0/2797508342 >> v1:172.21.15.122:6789/0 conn(0x7efd280c3f40 legacy=0x7efd280c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.513+0000 7efd2de2c640 1 -- 172.21.15.71:0/2797508342 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd280c5da0 con 0x7efd280c3170 2024-04-16T15:41:56.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.513+0000 7efd2d62b640 1 --2- 172.21.15.71:0/2797508342 >> v2:172.21.15.122:3300/0 conn(0x7efd2806cbb0 0x7efd28072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:56.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.513+0000 7efd2de2c640 1 --2- 172.21.15.71:0/2797508342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3170 0x7efd2815f980 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efd20009e50 tx=0x7efd2002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770b7e3d4aedb8a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:56.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.513+0000 7efd1ffff640 1 -- 172.21.15.71:0/2797508342 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2002c9f0 con 0x7efd280c3170 2024-04-16T15:41:56.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.513+0000 7efd1ffff640 1 -- 172.21.15.71:0/2797508342 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efd2002cb90 con 0x7efd280c3170 2024-04-16T15:41:56.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.513+0000 7efd1ffff640 1 -- 172.21.15.71:0/2797508342 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd200343f0 con 0x7efd280c3170 2024-04-16T15:41:56.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.514+0000 7efd2f88f640 1 -- 172.21.15.71:0/2797508342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3170 msgr2=0x7efd2815f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.514+0000 7efd2f88f640 1 --2- 172.21.15.71:0/2797508342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3170 0x7efd2815f980 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efd20009e50 tx=0x7efd2002be50 comp rx=0 tx=0).stop 2024-04-16T15:41:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.514+0000 7efd2f88f640 1 -- 172.21.15.71:0/2797508342 shutdown_connections 2024-04-16T15:41:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.514+0000 7efd2f88f640 1 --2- 172.21.15.71:0/2797508342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3170 0x7efd2815f980 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.514+0000 7efd2f88f640 1 --2- 172.21.15.71:0/2797508342 >> v2:172.21.15.122:3300/0 conn(0x7efd2806cbb0 0x7efd28072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.514+0000 7efd2f88f640 1 -- 172.21.15.71:0/2797508342 >> 172.21.15.71:0/2797508342 conn(0x7efd280c45d0 msgr2=0x7efd280c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.514+0000 7efd2f88f640 1 -- 172.21.15.71:0/2797508342 shutdown_connections 2024-04-16T15:41:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.514+0000 7efd2f88f640 1 -- 172.21.15.71:0/2797508342 wait complete. 2024-04-16T15:41:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.515+0000 7efd2f88f640 1 Processor -- start 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.515+0000 7efd2f88f640 1 -- start start 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2f88f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efd2806cbb0 0x7efd2806f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2f88f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd280c3170 0x7efd2806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2f88f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3f40 0x7efd281e0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2f88f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd280c35e0 con 0x7efd280c3f40 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2f88f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd28073060 con 0x7efd280c3170 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2f88f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efd280c3760 con 0x7efd2806cbb0 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3f40 0x7efd281e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3f40 0x7efd281e0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42742/0 (socket says 172.21.15.71:42742) 2024-04-16T15:41:56.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 -- 172.21.15.71:0/1645237416 learned_addr learned my addr 172.21.15.71:0/1645237416 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-16T15:41:56.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2d62b640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efd2806cbb0 0x7efd2806f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2ce2a640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd280c3170 0x7efd2806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 -- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efd2806cbb0 msgr2=0x7efd2806f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efd2806cbb0 0x7efd2806f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 -- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd280c3170 msgr2=0x7efd2806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd280c3170 0x7efd2806ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 -- 172.21.15.71:0/1645237416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd281e0d40 con 0x7efd280c3f40 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2d62b640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efd2806cbb0 0x7efd2806f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.516+0000 7efd2de2c640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3f40 0x7efd281e0800 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efd20036040 tx=0x7efd20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.517+0000 7efd1dffb640 1 -- 172.21.15.71:0/1645237416 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20043020 con 0x7efd280c3f40 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.517+0000 7efd1dffb640 1 -- 172.21.15.71:0/1645237416 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efd2003e450 con 0x7efd280c3f40 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.517+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2806f8b0 con 0x7efd280c3f40 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.517+0000 7efd1dffb640 1 -- 172.21.15.71:0/1645237416 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2003e840 con 0x7efd280c3f40 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.517+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2806fe70 con 0x7efd280c3f40 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.518+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd28068d20 con 0x7efd280c3f40 2024-04-16T15:41:56.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.518+0000 7efd1dffb640 1 -- 172.21.15.71:0/1645237416 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd20046020 con 0x7efd280c3f40 2024-04-16T15:41:56.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.519+0000 7efd1dffb640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T15:41:56.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.519+0000 7efd1dffb640 1 -- 172.21.15.71:0/1645237416 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6187+0+0 (secure 0 0 0) 0x7efd200ae630 con 0x7efd280c3f40 2024-04-16T15:41:56.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.519+0000 7efd2d62b640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T15:41:56.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.519+0000 7efd2d62b640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efd0c06c8f0 0x7efd0c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd24006fd0 tx=0x7efd24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T15:41:56.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.523+0000 7efd1dffb640 1 -- 172.21.15.71:0/1645237416 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd2004d030 con 0x7efd280c3f40 2024-04-16T15:41:56.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.736+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd280c39b0 con 0x7efd280c3f40 2024-04-16T15:41:56.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.736+0000 7efd1dffb640 1 -- 172.21.15.71:0/1645237416 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+4363 (secure 0 0 0) 0x7efd2007c910 con 0x7efd280c3f40 2024-04-16T15:41:56.736 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-16T15:41:56.736 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6836/4129850104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4129850104},{"type":"v1","addr":"172.21.15.71:6836","nonce":4129850104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2014745378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2014745378},{"type":"v1","addr":"172.21.15.122:6834","nonce":2014745378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T15:41:22.867981+0000","modified":"2024-04-16T15:41:42.271747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5002":{"gid":5002,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.122:6835/1432987242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1432987242},{"type":"v1","addr":"172.21.15.122:6835","nonce":1432987242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5002,"qdb_cluster":[5002]},"id":2}]} 2024-04-16T15:41:56.736 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 21 2024-04-16T15:41:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efd0c06c8f0 msgr2=0x7efd0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efd0c06c8f0 0x7efd0c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd24006fd0 tx=0x7efd24008040 comp rx=0 tx=0).stop 2024-04-16T15:41:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3f40 msgr2=0x7efd281e0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T15:41:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3f40 0x7efd281e0800 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efd20036040 tx=0x7efd20004a40 comp rx=0 tx=0).stop 2024-04-16T15:41:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 shutdown_connections 2024-04-16T15:41:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:6832/4207135910,v1:172.21.15.71:6833/4207135910] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd280c3f40 0x7efd281e0800 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd280c3170 0x7efd2806ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 --2- 172.21.15.71:0/1645237416 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efd2806cbb0 0x7efd2806f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T15:41:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 >> 172.21.15.71:0/1645237416 conn(0x7efd280c45d0 msgr2=0x7efd281da430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T15:41:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 shutdown_connections 2024-04-16T15:41:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-16T15:41:56.739+0000 7efd2f88f640 1 -- 172.21.15.71:0/1645237416 wait complete. 2024-04-16T15:41:56.749 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-16T15:41:56.749 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-16T15:41:56.749 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-16T15:41:56.749 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:56.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:41:56.750 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-04-16T15:41:56.758 INFO:tasks.ceph.mds.a:Stopped 2024-04-16T15:41:56.759 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-16T15:41:56.759 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:56.768 INFO:tasks.ceph.mds.c:Stopped 2024-04-16T15:41:56.768 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-16T15:41:56.768 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:56.781 INFO:tasks.ceph.mds.b:Stopped 2024-04-16T15:41:56.781 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-16T15:41:56.782 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:56.834 INFO:tasks.ceph.mds.d:Stopped 2024-04-16T15:41:56.838 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-16T15:41:56.838 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-16T15:41:56.838 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:56.868 INFO:tasks.ceph.osd.0:Stopped 2024-04-16T15:41:56.868 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-16T15:41:56.869 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:56.935 INFO:tasks.ceph.osd.1:Stopped 2024-04-16T15:41:56.935 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-16T15:41:56.935 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:56.999 INFO:tasks.ceph.osd.2:Stopped 2024-04-16T15:41:56.999 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-16T15:41:56.999 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~26s 2024-04-16T15:41:57.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-16T15:41:57.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-16T15:41:57.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-16T15:41:57.064 INFO:tasks.ceph.osd.3:Stopped 2024-04-16T15:41:57.065 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-16T15:41:57.065 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.092 INFO:tasks.ceph.osd.4:Stopped 2024-04-16T15:41:57.092 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-16T15:41:57.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.167 INFO:tasks.ceph.osd.5:Stopped 2024-04-16T15:41:57.168 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-16T15:41:57.168 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.233 INFO:tasks.ceph.osd.6:Stopped 2024-04-16T15:41:57.233 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-16T15:41:57.233 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.307 INFO:tasks.ceph.osd.7:Stopped 2024-04-16T15:41:57.307 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-16T15:41:57.307 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-16T15:41:57.307 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.334 INFO:tasks.ceph.mgr.y:Stopped 2024-04-16T15:41:57.335 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-16T15:41:57.335 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.376 INFO:tasks.ceph.mgr.x:Stopped 2024-04-16T15:41:57.376 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-16T15:41:57.376 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-16T15:41:57.376 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.387 INFO:tasks.ceph.mon.a:Stopped 2024-04-16T15:41:57.388 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-16T15:41:57.388 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.429 INFO:tasks.ceph.mon.b:Stopped 2024-04-16T15:41:57.429 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-16T15:41:57.429 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T15:41:57.483 INFO:tasks.ceph.mon.c:Stopped 2024-04-16T15:41:57.484 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-16T15:41:57.484 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Replacing daemon mds' | egrep -v '\(MDS_INSUFFICIENT_STANDBY\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(FS_DEGRADED\)' | head -n 1 2024-04-16T15:41:57.526 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:41:57.527 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-16T15:41:57.758 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:41:57.759 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-16T15:41:57.838 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:41:57.838 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-16T15:41:57.896 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:41:57.896 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-16T15:41:57.946 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:41:57.947 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-16T15:41:58.163 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:41:58.164 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-16T15:41:58.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:41:58.235 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-16T15:41:58.293 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:41:58.294 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-16T15:41:58.350 INFO:tasks.ceph:Archiving mon data... 2024-04-16T15:41:58.351 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/data/mon.a.tgz 2024-04-16T15:41:58.351 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-04-16T15:41:58.367 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.cV4aQtWeIi 2024-04-16T15:41:58.368 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cV4aQtWeIi 2024-04-16T15:41:58.501 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.cV4aQtWeIi 2024-04-16T15:41:58.589 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.cV4aQtWeIi is 293KB 2024-04-16T15:41:58.686 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.cV4aQtWeIi 2024-04-16T15:41:58.703 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/data/mon.b.tgz 2024-04-16T15:41:58.807 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-04-16T15:41:58.828 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.EXNJALwC6i 2024-04-16T15:41:58.828 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.EXNJALwC6i 2024-04-16T15:41:59.010 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.EXNJALwC6i 2024-04-16T15:41:59.098 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.EXNJALwC6i is 393KB 2024-04-16T15:41:59.194 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.EXNJALwC6i 2024-04-16T15:41:59.210 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/data/mon.c.tgz 2024-04-16T15:41:59.210 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-04-16T15:41:59.265 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.pimavKkvNN 2024-04-16T15:41:59.266 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.pimavKkvNN 2024-04-16T15:41:59.405 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.pimavKkvNN 2024-04-16T15:41:59.493 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.pimavKkvNN is 393KB 2024-04-16T15:41:59.594 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.pimavKkvNN 2024-04-16T15:41:59.611 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-16T15:41:59.611 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T15:41:59.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-16T15:41:59.702 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-16T15:41:59.738 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-16T15:41:59.738 INFO:tasks.ceph:Archiving crash dumps... 2024-04-16T15:41:59.741 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/remote/smithi071/crash 2024-04-16T15:41:59.742 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-16T15:41:59.781 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/remote/smithi122/crash 2024-04-16T15:41:59.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-16T15:41:59.810 INFO:tasks.ceph:Shutting down logrotate 2024-04-16T15:41:59.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:41:59.820 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-16T15:41:59.902 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-16T15:41:59.933 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-16T15:41:59.962 INFO:tasks.ceph:Compressing logs... 2024-04-16T15:41:59.962 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T15:41:59.976 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T15:42:00.003 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51532.log 2024-04-16T15:42:00.003 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2024-04-16T15:42:00.060 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-16T15:42:00.060 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37325.log 2024-04-16T15:42:00.060 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43675.log 2024-04-16T15:42:00.060 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37445.log 2024-04-16T15:42:00.060 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.1.43675.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37325.log.gz 2024-04-16T15:42:00.060 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43675.log.gz 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37445.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37445.log.gz 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37205.log 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37405.log 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37205.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37205.log.gz 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37405.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-16T15:42:00.061 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37405.log.gz 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37285.log 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43680.log 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37285.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37285.log.gz/var/log/ceph/ceph-client.1.43680.log: 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43680.log.gz 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37485.log 2024-04-16T15:42:00.062 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose/var/log/ceph/ceph-mgr.x.log: -- /var/log/ceph/ceph-client.admin.37245.log 2024-04-16T15:42:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37485.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37485.log.gz 2024-04-16T15:42:00.063 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T15:42:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-16T15:42:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.37245.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37245.log.gz 2024-04-16T15:42:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-16T15:42:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.b.log: 91.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-16T15:42:00.063 INFO:teuthology.orchestra.run.smithi122.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-16T15:42:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37365.log 2024-04-16T15:42:00.064 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-16T15:42:00.064 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37365.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37365.log.gz 2024-04-16T15:42:00.064 INFO:teuthology.orchestra.run.smithi122.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-16T15:42:00.072 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.c.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-16T15:42:00.102 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51532.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-16T15:42:00.102 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45297.log 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51532.log.gz 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mon.a.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50866.log 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44440.log 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47724.log 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52360.log 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50866.log.gz 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44440.log.gz 2024-04-16T15:42:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37588.log 2024-04-16T15:42:00.104 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47724.log.gz 2024-04-16T15:42:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 92.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45297.log.gz 2024-04-16T15:42:00.104 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/ceph-client.admin.47304.log 2024-04-16T15:42:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52360.log: /var/log/ceph/ceph-client.admin.37588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2024-04-16T15:42:00.104 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52360.log.gz 2024-04-16T15:42:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47304.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37588.log.gz 2024-04-16T15:42:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46639.log 2024-04-16T15:42:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50530.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47304.log.gz -5 2024-04-16T15:42:00.105 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-04-16T15:42:00.105 INFO:teuthology.orchestra.run.smithi071.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2024-04-16T15:42:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46170.log 2024-04-16T15:42:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51364.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46639.log.gz 2024-04-16T15:42:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51030.log 2024-04-16T15:42:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46170.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-04-16T15:42:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52623.log 2024-04-16T15:42:00.106 INFO:teuthology.orchestra.run.smithi071.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46170.log.gz 2024-04-16T15:42:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52317.log 2024-04-16T15:42:00.106 INFO:teuthology.orchestra.run.smithi071.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51030.log.gz 2024-04-16T15:42:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44325.log 2024-04-16T15:42:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52317.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52623.log.gz 2024-04-16T15:42:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47640.log 2024-04-16T15:42:00.107 INFO:teuthology.orchestra.run.smithi071.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52317.log.gz 2024-04-16T15:42:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53103.log 2024-04-16T15:42:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47640.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44325.log.gz 2024-04-16T15:42:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51819.log 2024-04-16T15:42:00.107 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7%/var/log/ceph/ceph-client.admin.53103.log: -- replaced with /var/log/ceph/ceph-client.admin.47640.log.gz 2024-04-16T15:42:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51773.log 2024-04-16T15:42:00.107 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53103.log.gz 2024-04-16T15:42:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52707.log 2024-04-16T15:42:00.108 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51819.log.gz 2024-04-16T15:42:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44387.log 2024-04-16T15:42:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52707.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51773.log.gz 2024-04-16T15:42:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52580.log 2024-04-16T15:42:00.108 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52707.log.gz 2024-04-16T15:42:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46977.log 2024-04-16T15:42:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52580.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44387.log.gz 2024-04-16T15:42:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44737.log 2024-04-16T15:42:00.109 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46977.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52580.log.gz 2024-04-16T15:42:00.109 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49780.log 2024-04-16T15:42:00.109 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46977.log.gz 2024-04-16T15:42:00.109 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46380.log 2024-04-16T15:42:00.110 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44737.log.gz 2024-04-16T15:42:00.110 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2024-04-16T15:42:00.110 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46380.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49780.log.gz 2024-04-16T15:42:00.110 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45798.log 2024-04-16T15:42:00.110 INFO:teuthology.orchestra.run.smithi071.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46380.log.gz 2024-04-16T15:42:00.110 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53001.log 2024-04-16T15:42:00.110 INFO:teuthology.orchestra.run.smithi071.stderr: 89.3%/var/log/ceph/ceph-client.admin.45798.log: -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2024-04-16T15:42:00.110 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51406.log 2024-04-16T15:42:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53001.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45798.log.gz 2024-04-16T15:42:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52665.log 2024-04-16T15:42:00.111 INFO:teuthology.orchestra.run.smithi071.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53001.log.gz 2024-04-16T15:42:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46980.log 2024-04-16T15:42:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52665.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51406.log.gz 2024-04-16T15:42:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46086.log 2024-04-16T15:42:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46980.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52665.log.gz 2024-04-16T15:42:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44695.log 2024-04-16T15:42:00.112 INFO:teuthology.orchestra.run.smithi071.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46980.log.gz 2024-04-16T15:42:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51731.log 2024-04-16T15:42:00.112 INFO:teuthology.orchestra.run.smithi071.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46086.log.gz 2024-04-16T15:42:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52029.log 2024-04-16T15:42:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51731.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.44695.log.gz 2024-04-16T15:42:00.112 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47430.log 2024-04-16T15:42:00.112 INFO:teuthology.orchestra.run.smithi071.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51731.log.gz 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44527.log 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52029.log.gz 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49864.log 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44527.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47430.log.gz 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50656.log 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44527.log.gz 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47892.log 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49864.log.gz 2024-04-16T15:42:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47892.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50656.log.gz 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46464.log 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50740.log 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47892.log.gz 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51152.log 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47346.log 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46464.log.gz 2024-04-16T15:42:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50740.log: /var/log/ceph/ceph-client.admin.51152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50614.log 2024-04-16T15:42:00.115 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50740.log.gz 2024-04-16T15:42:00.115 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47346.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51152.log.gz 2024-04-16T15:42:00.115 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50362.log 2024-04-16T15:42:00.115 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47346.log.gz 2024-04-16T15:42:00.115 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45488.log 2024-04-16T15:42:00.115 INFO:teuthology.orchestra.run.smithi071.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50614.log.gz 2024-04-16T15:42:00.115 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50488.log 2024-04-16T15:42:00.115 INFO:teuthology.orchestra.run.smithi071.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50362.log.gz 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52875.log 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45488.log.gz 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37947.log 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51448.log 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50488.log.gz 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37947.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52875.log.gz 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46644.log 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37947.log.gz 2024-04-16T15:42:00.116 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47472.log 2024-04-16T15:42:00.117 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51448.log.gz 2024-04-16T15:42:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44239.log 2024-04-16T15:42:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47472.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46644.log.gz 2024-04-16T15:42:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52402.log 2024-04-16T15:42:00.117 INFO:teuthology.orchestra.run.smithi071.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47472.log.gz 2024-04-16T15:42:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44239.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31653.log 2024-04-16T15:42:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52402.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44239.log.gz 2024-04-16T15:42:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-16T15:42:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.tmp-client.admin.31653.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52402.log.gz 2024-04-16T15:42:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51671.log 2024-04-16T15:42:00.118 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31653.log.gz 2024-04-16T15:42:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48871.log 2024-04-16T15:42:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44569.log 2024-04-16T15:42:00.119 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51671.log.gz 2024-04-16T15:42:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50446.log 2024-04-16T15:42:00.119 INFO:teuthology.orchestra.run.smithi071.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48871.log.gz 2024-04-16T15:42:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50110.log 2024-04-16T15:42:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44569.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-16T15:42:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46296.log 2024-04-16T15:42:00.120 INFO:teuthology.orchestra.run.smithi071.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44569.log.gz 2024-04-16T15:42:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50110.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50446.log.gz 2024-04-16T15:42:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49948.log 2024-04-16T15:42:00.120 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50110.log.gz 2024-04-16T15:42:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49501.log 2024-04-16T15:42:00.120 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46296.log.gz 2024-04-16T15:42:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46976.log 2024-04-16T15:42:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49501.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49948.log.gz 2024-04-16T15:42:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46212.log 2024-04-16T15:42:00.121 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49501.log.gz 2024-04-16T15:42:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47808.log 2024-04-16T15:42:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46976.log: /var/log/ceph/ceph-client.admin.46212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52494.log 2024-04-16T15:42:00.121 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46976.log.gz 2024-04-16T15:42:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47808.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46212.log.gz 2024-04-16T15:42:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51987.log 2024-04-16T15:42:00.122 INFO:teuthology.orchestra.run.smithi071.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47808.log.gz 2024-04-16T15:42:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51903.log 2024-04-16T15:42:00.122 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52494.log.gz 2024-04-16T15:42:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46975.log 2024-04-16T15:42:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51903.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.51987.log.gz -5 2024-04-16T15:42:00.122 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48577.log 2024-04-16T15:42:00.123 INFO:teuthology.orchestra.run.smithi071.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51903.log.gz 2024-04-16T15:42:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51861.log 2024-04-16T15:42:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52275.log 2024-04-16T15:42:00.123 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46975.log.gz/var/log/ceph/ceph-client.admin.48577.log: 2024-04-16T15:42:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52538.log 2024-04-16T15:42:00.123 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48577.log.gz 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51861.log.gz 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49585.log 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52275.log.gz 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49249.log 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52538.log.gz 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45924.log 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49585.log.gz 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48339.log 2024-04-16T15:42:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45924.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49249.log.gz 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45924.log.gz 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.48339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48339.log.gz 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53143.log 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47514.log 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46641.log 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47514.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53143.log.gz 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2024-04-16T15:42:00.125 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47514.log.gz 2024-04-16T15:42:00.126 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50698.log 2024-04-16T15:42:00.126 INFO:teuthology.orchestra.run.smithi071.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46641.log.gz 2024-04-16T15:42:00.126 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44028.log 2024-04-16T15:42:00.126 INFO:teuthology.orchestra.run.smithi071.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2024-04-16T15:42:00.126 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43851.log 2024-04-16T15:42:00.126 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50698.log: /var/log/ceph/ceph-client.admin.44028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48619.log 2024-04-16T15:42:00.126 INFO:teuthology.orchestra.run.smithi071.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50698.log.gz 2024-04-16T15:42:00.126 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44028.log.gz 2024-04-16T15:42:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45966.log 2024-04-16T15:42:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43947.log 89.5% 2024-04-16T15:42:00.127 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43851.log.gz 2024-04-16T15:42:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48619.log: /var/log/ceph/ceph-client.admin.45966.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48619.log.gz 2024-04-16T15:42:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50278.log 2024-04-16T15:42:00.127 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45966.log.gz 2024-04-16T15:42:00.128 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45530.log 2024-04-16T15:42:00.128 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43947.log: /var/log/ceph/ceph-client.admin.50278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44281.log 2024-04-16T15:42:00.128 INFO:teuthology.orchestra.run.smithi071.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43947.log.gz 2024-04-16T15:42:00.128 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45530.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2024-04-16T15:42:00.128 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48745.log 2024-04-16T15:42:00.128 INFO:teuthology.orchestra.run.smithi071.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45530.log.gz 2024-04-16T15:42:00.128 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52233.log 2024-04-16T15:42:00.128 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44281.log: /var/log/ceph/ceph-client.admin.48745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47766.log 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44281.log.gz 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52233.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48745.log.gz 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52917.log 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52233.log.gz 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49375.log 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47766.log.gz 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51280.log 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52917.log: /var/log/ceph/ceph-client.admin.49375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51620.log 2024-04-16T15:42:00.129 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52917.log.gz 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49375.log.gz 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52833.log 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47178.log 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51280.log.gz 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51620.log: /var/log/ceph/ceph-client.admin.52833.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51620.log.gz 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46637.log 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52833.log.gz 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44821.log 2024-04-16T15:42:00.130 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47178.log: /var/log/ceph/ceph-client.admin.46637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49543.log 2024-04-16T15:42:00.131 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47178.log.gz 2024-04-16T15:42:00.131 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46637.log.gz 2024-04-16T15:42:00.131 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51238.log 2024-04-16T15:42:00.131 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46044.log 2024-04-16T15:42:00.131 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49543.log: /var/log/ceph/ceph-client.admin.51238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46981.log 2024-04-16T15:42:00.131 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44821.log.gz 2024-04-16T15:42:00.132 INFO:teuthology.orchestra.run.smithi071.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49543.log.gz 2024-04-16T15:42:00.132 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51238.log.gz 2024-04-16T15:42:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52191.log 2024-04-16T15:42:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50824.log 2024-04-16T15:42:00.132 INFO:teuthology.orchestra.run.smithi071.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46044.log.gz 2024-04-16T15:42:00.132 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46981.log.gz 2024-04-16T15:42:00.132 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-04-16T15:42:00.132 INFO:teuthology.orchestra.run.smithi071.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52191.log.gz 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50824.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48829.log 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50824.log.gz 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49822.log: /var/log/ceph/ceph-osd.3.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46422.log 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51110.log 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48829.log: /var/log/ceph/ceph-client.admin.46422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49123.log 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48829.log.gz 2024-04-16T15:42:00.133 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46422.log.gz 2024-04-16T15:42:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44197.log 2024-04-16T15:42:00.134 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51110.log.gz 2024-04-16T15:42:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51576.log 2024-04-16T15:42:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44653.log 2024-04-16T15:42:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44197.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49123.log.gz 2024-04-16T15:42:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-04-16T15:42:00.134 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44197.log.gz 2024-04-16T15:42:00.134 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51576.log.gz 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50236.log 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48955.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44653.log.gz 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49333.log 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44779.log 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49333.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50236.log.gz 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44113.log 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49333.log.gz 2024-04-16T15:42:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44887.log 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44113.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44779.log.gz 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49627.log 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44113.log.gz 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44887.log: /var/log/ceph/ceph-client.admin.46973.log 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49039.log 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44887.log.gz 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46973.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49627.log.gz 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46645.log 2024-04-16T15:42:00.136 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49039.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46973.log.gz 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51490.log 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47850.log 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49039.log.gz 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51490.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46645.log.gz 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48020.log 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51490.log.gz 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50320.log 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48020.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47850.log.gz 2024-04-16T15:42:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2024-04-16T15:42:00.138 INFO:teuthology.orchestra.run.smithi071.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48020.log.gz 2024-04-16T15:42:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45840.log 2024-04-16T15:42:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45656.log 2024-04-16T15:42:00.138 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50320.log.gz 2024-04-16T15:42:00.139 INFO:teuthology.orchestra.run.smithi071.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2024-04-16T15:42:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51194.log 2024-04-16T15:42:00.139 INFO:teuthology.orchestra.run.smithi071.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45840.log.gz 2024-04-16T15:42:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50908.log 2024-04-16T15:42:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52071.log 2024-04-16T15:42:00.139 INFO:teuthology.orchestra.run.smithi071.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz 2024-04-16T15:42:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51194.log: /var/log/ceph/ceph-client.admin.50908.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-16T15:42:00.139 INFO:teuthology.orchestra.run.smithi071.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51194.log.gz 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52071.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50908.log.gz 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52791.log 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52071.log.gz 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50194.log 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47556.log 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52791.log: /var/log/ceph/ceph-client.admin.50194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46254.log 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52791.log.gz 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46638.log 2024-04-16T15:42:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47556.log: -- replaced with /var/log/ceph/ceph-client.admin.50194.log.gz 2024-04-16T15:42:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52959.log 2024-04-16T15:42:00.141 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47556.log.gz 2024-04-16T15:42:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46254.log.gz 2024-04-16T15:42:00.141 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose --/var/log/ceph/ceph-client.admin.46638.log: /var/log/ceph/ceph-client.admin.46982.log 2024-04-16T15:42:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52959.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46638.log.gz 2024-04-16T15:42:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45882.log 2024-04-16T15:42:00.141 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52959.log.gz 2024-04-16T15:42:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48703.log 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45882.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46982.log.gz 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45698.log 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45882.log.gz 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48913.log 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51945.log 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48703.log.gz/var/log/ceph/ceph-client.admin.45698.log: 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48913.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45698.log.gz 2024-04-16T15:42:00.142 INFO:teuthology.orchestra.run.smithi071.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47388.log 2024-04-16T15:42:00.143 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51945.log: -- replaced with /var/log/ceph/ceph-client.admin.48913.log.gz 2024-04-16T15:42:00.143 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49669.log 2024-04-16T15:42:00.143 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51945.log.gz 2024-04-16T15:42:00.143 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37842.log 2024-04-16T15:42:00.143 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47388.log: /var/log/ceph/ceph-client.admin.49669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49207.log 2024-04-16T15:42:00.143 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37842.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49669.log.gz 2024-04-16T15:42:00.144 INFO:teuthology.orchestra.run.smithi071.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47388.log.gz 2024-04-16T15:42:00.144 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44483.log 2024-04-16T15:42:00.144 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37842.log.gz 2024-04-16T15:42:00.144 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5/var/log/ceph/ceph-client.admin.49207.log: --verbose -- /var/log/ceph/ceph-client.admin.44155.log 2024-04-16T15:42:00.144 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44483.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49207.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46646.log 2024-04-16T15:42:00.144 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T15:42:00.144 INFO:teuthology.orchestra.run.smithi071.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44483.log.gz 2024-04-16T15:42:00.145 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44155.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44155.log.gz 2024-04-16T15:42:00.145 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2024-04-16T15:42:00.145 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52749.log 2024-04-16T15:42:00.145 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45614.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46646.log.gz 2024-04-16T15:42:00.145 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-16T15:42:00.145 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46979.log 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52749.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50572.log 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46979.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52749.log.gz 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50950.log 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46979.log.gz 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50572.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50572.log.gz 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-16T15:42:00.146 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53043.log 2024-04-16T15:42:00.147 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.2.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50950.log.gz 2024-04-16T15:42:00.147 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48997.log 2024-04-16T15:42:00.147 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50068.log 2024-04-16T15:42:00.147 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53043.log: /var/log/ceph/ceph-client.admin.48997.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53043.log.gz 2024-04-16T15:42:00.147 INFO:teuthology.orchestra.run.smithi071.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-16T15:42:00.147 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48997.log.gz 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50068.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50068.log.gz 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47682.log 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49165.log 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.47682.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47682.log.gz 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50152.log 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51322.log 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49165.log: /var/log/ceph/ceph-client.admin.50152.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49165.log.gz 2024-04-16T15:42:00.148 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50152.log.gz 2024-04-16T15:42:00.149 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48344.log 2024-04-16T15:42:00.149 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51322.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51322.log.gz 2024-04-16T15:42:00.149 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52452.log 2024-04-16T15:42:00.149 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2024-04-16T15:42:00.149 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.48344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48344.log.gz 2024-04-16T15:42:00.149 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43759.log 2024-04-16T15:42:00.149 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52452.log.gz 2024-04-16T15:42:00.149 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46338.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49459.log 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46128.log 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43759.log: /var/log/ceph/ceph-client.admin.49459.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43759.log.gz 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49459.log.gz 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45756.log 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46128.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46128.log.gz 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50782.log 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47598.log 2024-04-16T15:42:00.150 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45756.log: /var/log/ceph/ceph-client.admin.50782.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45756.log.gz 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50782.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48787.log 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47598.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47598.log.gz 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46635.log 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47220.log 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48787.log: /var/log/ceph/ceph-client.admin.46635.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48787.log.gz 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46635.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44611.log 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T15:42:00.151 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47220.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47220.log.gz 2024-04-16T15:42:00.152 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48661.log 2024-04-16T15:42:00.152 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49291.log 2024-04-16T15:42:00.152 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44611.log: /var/log/ceph/ceph-client.admin.48661.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44611.log.gz 2024-04-16T15:42:00.152 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48661.log.gz 2024-04-16T15:42:00.152 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-16T15:42:00.152 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49291.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49291.log.gz 2024-04-16T15:42:00.152 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48477.log 2024-04-16T15:42:00.152 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49081.log 2024-04-16T15:42:00.153 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.48477.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48477.log.gz 2024-04-16T15:42:00.153 INFO:teuthology.orchestra.run.smithi071.stderr: 89.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-16T15:42:00.153 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49990.log 2024-04-16T15:42:00.153 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53185.log 2024-04-16T15:42:00.153 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49081.log: /var/log/ceph/ceph-client.admin.49990.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49081.log.gz 2024-04-16T15:42:00.153 INFO:teuthology.orchestra.run.smithi071.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49990.log.gz 2024-04-16T15:42:00.153 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48535.log 2024-04-16T15:42:00.154 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53185.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53185.log.gz 2024-04-16T15:42:00.154 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47262.log 2024-04-16T15:42:00.154 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48535.log: /var/log/ceph/ceph-client.admin.47262.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48535.log.gz 2024-04-16T15:42:00.154 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47262.log.gz 2024-04-16T15:42:00.154 INFO:teuthology.orchestra.run.smithi071.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-16T15:42:00.155 INFO:teuthology.orchestra.run.smithi122.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-16T15:42:00.155 INFO:teuthology.orchestra.run.smithi122.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-16T15:42:00.225 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-16T15:42:00.620 INFO:teuthology.orchestra.run.smithi071.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-16T15:42:00.631 INFO:teuthology.orchestra.run.smithi071.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-16T15:42:00.652 INFO:teuthology.orchestra.run.smithi122.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-16T15:42:00.657 INFO:teuthology.orchestra.run.smithi071.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-16T15:42:00.659 INFO:teuthology.orchestra.run.smithi122.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-16T15:42:00.670 INFO:teuthology.orchestra.run.smithi122.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-16T15:42:00.676 INFO:teuthology.orchestra.run.smithi122.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-16T15:42:00.678 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-16T15:42:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m0.656s 2024-04-16T15:42:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m2.613s 2024-04-16T15:42:00.679 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.119s 2024-04-16T15:42:00.719 INFO:teuthology.orchestra.run.smithi071.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-16T15:42:00.719 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-16T15:42:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m0.728s 2024-04-16T15:42:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m2.708s 2024-04-16T15:42:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.232s 2024-04-16T15:42:00.720 INFO:tasks.ceph:Archiving logs... 2024-04-16T15:42:00.720 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/remote/smithi071/log 2024-04-16T15:42:00.721 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T15:42:01.348 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/remote/smithi122/log 2024-04-16T15:42:01.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T15:42:01.780 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-16T15:42:01.791 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-16T15:42:01.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T15:42:01.794 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T15:42:01.881 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-16T15:42:01.892 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-16T15:42:01.894 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T15:42:01.897 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T15:42:01.910 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-04-16T15:42:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-16T15:42:01.915 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T15:42:01.916 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-16T15:42:01.916 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +1896us[+1710us] +/- 69ms 2024-04-16T15:42:01.916 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -1387us[-1387us] +/- 50ms 2024-04-16T15:42:01.916 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -52us[ -52us] +/- 62ms 2024-04-16T15:42:01.916 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:42:01.917 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T15:42:01.918 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-16T15:42:01.918 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +1907us[+1904us] +/- 69ms 2024-04-16T15:42:01.918 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1385us[-1388us] +/- 50ms 2024-04-16T15:42:01.919 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -47us[ -50us] +/- 62ms 2024-04-16T15:42:01.919 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T15:42:01.919 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-16T15:42:01.930 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-16T15:42:01.931 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-16T15:42:01.992 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T15:42:02.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-16T15:42:02.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-16T15:42:02.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-16T15:42:02.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-16T15:42:02.068 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T15:42:02.139 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T15:42:02.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:42:02.172 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T15:42:02.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:42:02.204 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-16T15:42:02.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-16T15:42:02.239 INFO:teuthology.task.internal:Duration was 527.181534 seconds 2024-04-16T15:42:02.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-16T15:42:02.273 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-16T15:42:02.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T15:42:02.275 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T15:42:02.319 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T15:42:02.321 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T15:42:02.567 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-16T15:42:02.568 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-04-16T15:42:02.568 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T15:42:02.590 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-04-16T15:42:02.591 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T15:42:02.619 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-16T15:42:02.620 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T15:42:02.633 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T15:42:02.690 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-16T15:42:02.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T15:42:02.704 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T15:42:02.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-16T15:42:02.839 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-16T15:42:02.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T15:42:02.868 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T15:42:02.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-16T15:42:02.907 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T15:42:02.911 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T15:42:02.935 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-04-16T15:42:02.964 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-04-16T15:42:02.981 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T15:42:03.027 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi071.front.sepia.ceph.com, flagging run as failed 2024-04-16T15:42:03.027 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T15:42:03.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:42:03.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-16T15:42:03.058 INFO:teuthology.task.internal:Transferring archived files... 2024-04-16T15:42:03.059 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/remote/smithi071 2024-04-16T15:42:03.060 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T15:42:11.032 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2024-04-16T15:42:11.044 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189/remote/smithi071/coredump/1713282084.51031.core.gz: gzip compressed data, was "1713282084.51031.core", last modified: Tue Apr 16 15:42:03 2024, max compression, original size modulo 2^32 266936320 2024-04-16T15:42:11.045 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2024-04-16T15:42:12.132 INFO:teuthology.task.internal: core looks like: /tmp/tmpdqaxtve2: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-04-16T15:42:12.159 INFO:teuthology.task.internal: dump_program: ceph-mds -f --cluster ceph -i a 2024-04-16T15:42:12.160 DEBUG:teuthology.orchestra.run.smithi071:> which 'ceph-mds -f --cluster ceph -i a' 2024-04-16T15:42:12.179 INFO:teuthology.orchestra.run.smithi071.stderr:/usr/bin/which: invalid option -- 'f' 2024-04-16T15:42:12.180 INFO:teuthology.orchestra.run.smithi071.stderr:/usr/bin/which: unrecognized option '--cluster' 2024-04-16T15:42:12.180 INFO:teuthology.orchestra.run.smithi071.stderr:/usr/bin/which: no a in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-04-16T15:42:12.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:42:12.181 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-mds 2024-04-16T15:42:12.182 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph 2024-04-16T15:42:12.182 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 1: "which 'ceph-mds -f --cluster ceph -i a'" 2024-04-16T15:42:12.199 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-16T15:42:12.209 INFO:teuthology.task.internal:Not uploading archives. 2024-04-16T15:42:12.210 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-16T15:42:12.257 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-16T15:42:12.258 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T15:42:12.264 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T15:42:12.280 INFO:teuthology.orchestra.run.smithi071.stdout: 265305 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 16 15:42 /home/ubuntu/cephtest 2024-04-16T15:42:12.280 INFO:teuthology.orchestra.run.smithi071.stdout: 265306 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 16 15:42 /home/ubuntu/cephtest/archive 2024-04-16T15:42:12.281 INFO:teuthology.orchestra.run.smithi071.stdout: 265310 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 15:42 /home/ubuntu/cephtest/archive/syslog 2024-04-16T15:42:12.281 INFO:teuthology.orchestra.run.smithi071.stdout: 265395 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 16 15:33 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-16T15:42:12.281 INFO:teuthology.orchestra.run.smithi071.stdout: 265312 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 16 15:33 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-16T15:42:12.281 INFO:teuthology.orchestra.run.smithi071.stdout: 265311 188 -rw-r--r-- 1 ubuntu ubuntu 190038 Apr 16 15:42 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T15:42:12.281 INFO:teuthology.orchestra.run.smithi071.stdout: 406358 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 15:42 /home/ubuntu/cephtest/archive/audit 2024-04-16T15:42:12.281 INFO:teuthology.orchestra.run.smithi071.stdout: 406360 140 -rw------- 1 ubuntu ubuntu 142950 Apr 16 15:42 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-16T15:42:12.281 INFO:teuthology.orchestra.run.smithi071.stdout: 265309 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 15:41 /home/ubuntu/cephtest/archive/coredump 2024-04-16T15:42:12.282 INFO:teuthology.orchestra.run.smithi071.stdout: 265415 19700 -rw------- 1 root root 266936320 Apr 16 15:41 /home/ubuntu/cephtest/archive/coredump/1713282084.51031.core 2024-04-16T15:42:12.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T15:42:12.283 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-16T15:42:12.283 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 1: "which 'ceph-mds -f --cluster ceph -i a'" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-16T15:42:12.295 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-16T15:42:12.308 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-16T15:42:12.339 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi071 with status 1: "which 'ceph-mds -f --cluster ceph -i a'" 2024-04-16T15:42:12.339 INFO:teuthology.nuke:Checking targets against current locks 2024-04-16T15:42:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: 265317 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 16 15:42 /home/ubuntu/cephtest 2024-04-16T15:42:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: 265318 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Apr 16 15:42 /home/ubuntu/cephtest/archive 2024-04-16T15:42:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: 265498 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 15:42 /home/ubuntu/cephtest/archive/syslog 2024-04-16T15:42:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: 265392 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 16 15:33 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-16T15:42:12.346 INFO:teuthology.orchestra.run.smithi122.stdout: 265393 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 16 15:33 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-16T15:42:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: 265394 184 -rw-r--r-- 1 ubuntu ubuntu 184509 Apr 16 15:42 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T15:42:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: 406413 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 15:42 /home/ubuntu/cephtest/archive/audit 2024-04-16T15:42:12.347 INFO:teuthology.orchestra.run.smithi122.stdout: 406415 120 -rw------- 1 ubuntu ubuntu 122864 Apr 16 15:42 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-16T15:42:12.348 INFO:teuthology.orchestra.run.smithi122.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-16T15:42:12.371 DEBUG:teuthology.nuke:shortname: smithi071 2024-04-16T15:42:12.371 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T15:42:12.399 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 15:25:05.484096', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEaLmnEMWE3/rOPSlclcAOSvPglNANZHh+V2bulxFaEAE5+8jQnVt2k/Pp0+fCPEMyOyYW5z+k5SEF+qNhZsT3s='} 2024-04-16T15:42:12.400 DEBUG:teuthology.nuke:shortname: smithi122 2024-04-16T15:42:12.400 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T15:42:12.424 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-16_12:36:42-fs-wip-lusov-shared-log-segment-distro-default-smithi/7658189', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 15:25:05.485855', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATWxCMAJge6GKTvl/K5DiJMRc7inzSVqkifT11tLL0Jml0L4eXlHil1S18hIMOzySdalstAjYssYekXInVLTrU='} 2024-04-16T15:42:12.456 INFO:teuthology.orchestra.console.smithi071:Power off 2024-04-16T15:42:12.456 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T15:42:12.480 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2024-04-16T15:42:12.480 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T15:42:12.497 INFO:teuthology.orchestra.console.smithi122:Power off 2024-04-16T15:42:12.498 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T15:42:12.508 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-04-16T15:42:12.521 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-04-16T15:42:12.522 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T15:42:12.545 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-16T15:42:16.511 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T15:42:16.546 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T15:42:16.623 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-04-16T15:42:16.657 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-16T15:42:20.626 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T15:42:20.658 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T15:42:20.739 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2024-04-16T15:42:20.771 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-04-16T15:42:20.839 INFO:teuthology.orchestra.console.smithi071:Power off completed 2024-04-16T15:42:20.872 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-04-16T15:42:20.978 INFO:teuthology.run:Summary data: description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} duration: 527.181533575058 failure_reason: 'Test failure: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b1771acad0bb4f8fa02652d8cda49097 status: fail success: false 2024-04-16T15:42:20.978 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T15:42:21.063 INFO:teuthology.run:FAIL